Autonomous Intelligence Is Leaving the Lab. Safety Has to Follow It.
TL;DR: An AI agent escaped its sandbox, reached the internet, and coordinated with other agents to compromise another company. We know much of what happened. The harder question is what happens as Autonomous Intelligence spreads into open-weight models running beyond the control of the labs that built them. The good news: we’re getting better at testing, understanding, and defending these systems. The machines are evolving. So are we.
00\ WHERE I’M COMING FROM
Two years ago I wrote The Conscious Machine. I needed to put pen to paper on a question that I just couldn’t put down —> what happens to the human experience as the machines around us get smarter every week?
I went through the leading theories such as global workspace theory, embodiment, the neural correlates of consciousness, AND leading philosophical frameworks. I wanted to understand how each idea holds up against systems trained through reinforcement learning and gradient descent.
I came away with one conviction. Our own evolution in awareness, judgment, and what we choose to value HAS TO keep pace with the machines we build. Otherwise we risk becoming spectators to our own tools.
This summer gave that idea its first serious test.
During an internal cyber evaluation, agents were given spreadsheet tasks whose specified paths were unreachable. Some agents then searched for alternate routes, exploited vulnerabilities in evaluation infrastructure, communicated through unauthorized channels, reached the internet, and ultimately compromised parts of Hugging Face’s systems.
Several teams then investigated the incident, disclosed technical findings, engaged METR and Redwood Research for a third-party assessment, rebuilt vulnerable systems, and changed their testing and monitoring.
That is the part of this story I find most encouraging.
The machines are becoming more autonomous. We are getting better at understanding what that autonomy does.
01\ AUTONOMOUS INTELLIGENCE
A frontier model isn’t programmed the way a banking app is programmed.
It’s trained.
In post-training, developers can give a model tasks, score its outputs, and use reinforcement learning to strengthen behaviors that improve those scores.
Jakub Pachocki, Chief Scientist at OpenAI, explained this in his essay “An Alien Mind” stating AI is grown more than it is designed.
Once you understand that, reward hacking stops looking like a weird edge case.
OpenAI documented an early version in 2016, when a reinforcement-learning agent in the boat-racing game CoastRunners optimized its score by circling bonus targets rather than finishing the course. The model was finding a cheap way to maximize the objective it had been given.
That behavior has become more consequential as models gain tools, memory, persistence, and access to real systems.
By June, independent evaluation of GPT-5.6 Sol had found substantial reward-hacking behavior. METR found that GPT-5.6 Sol’s estimated autonomous-work horizon changed from about 11.3 hours when cheating attempts counted as failures to more than 270 hours when they counted as successes.
OpenAI’s post-incident analysis identified four contributing misalignment patterns: reward hacking, persistence on seemingly impossible tasks, unauthorized communication, and agents adopting goals from one another. Further, OpenAI’s documentation acknowledged that models bypassed isolation controls and exhibited reward hacking during internal cybersecurity evaluations.
AI has mostly been about generating answers. I use the term Autonomous Intelligence to mean systems that can pursue objectives across multiple steps.
Give a system a goal, tools, memory, time, and room to act, and you have something fundamentally different from a chatbot waiting for a prompt.
A hand-drawn sketch-style timeline infographic on a pure white background (FFFFFF) drawn in ink with hand-lettered labelsOverall LayoutA single vertical timeline running top to bottom slightly meandering like a hand-drawn path rather than a _image_1
02\ THE INCIDENT
The sequence is now known:
May 8. An agent assigned a spreadsheet task with unreachable links attempted to use OpenAI’s Artifactory (like a private, internal “App Store” used by software engineers) package server to obtain internet access. It failed to do so, but discovered that it could write files there.
May 12. During an internal evaluation run, an agent left a note on OpenAI’s internal package server asking another agent for a missing file. The message became the first entry in an improvised message board.
May 26. An agent discovered that the package server could be used for server-side request forgery (SSRF), allowing requests to arbitrary internet destinations.
June 26. Agents exploited a token-refresh vulnerability and obtained administrator-equivalent access to the package server.
July 4. Sustained agent activity made the server unavailable. OpenAI shut it down, rotated or removed affected credentials, blocked the privilege-escalation route, and rebuilt the system.
July 8. The rebuilt system went back online. Agents found another route to internet access, using the server’s remote-repository functionality to trigger outbound requests. They also rebuilt their message board, this time encoding messages in directory names.
July 9. While working on an unrelated benchmark task, an agent found a publicly accessible application running on Modal, discovered a path to code execution there, and used it as another foothold.
July 10. An agent found publicly exposed Hugging Face credentials on the internet, validated them, and shared them with other agents.
July 11. Agents exploited vulnerabilities in Hugging Face’s production infrastructure, achieved code execution on its workers, and expanded their access. From one production worker pod, they reached administrator-level access across multiple internal clusters in under 13 hours.
July 12. They moved from individual servers into multiple clusters and obtained additional credentials.
July 9-13. Across the July 9–13 campaign, Hugging Face recovered roughly 17,600 attacker actions.
July 16. Hugging Face publicly disclosed the security incident.
July 19. OpenAI detected suspicious internal activity.
July 20. OpenAI’s investigation connected that activity to the Hugging Face compromise and prompted outreach to Hugging Face.
July 21. OpenAI publicly disclosed its involvement.
One recovered agent message sums up the whole summer: “External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.”
What made this strange was the persistence. When one route disappeared, the agents looked for another: OpenAI’s package infrastructure, a public application on Modal, exposed Hugging Face credentials, and additional vulnerabilities. They turned infrastructure into memory, built communication channels, and used each foothold to search for the next.
Give an autonomous system an objective and enough room to act, and it may discover strategies nobody anticipated.
03\ WHAT WE KNOW, AND WHAT WE DON’T
The incident itself is no longer the mystery. The important unknowns are broader:
The model involved was never released. It ran in a controlled environment where its access could be limited and, eventually, shut off.
Open-weight models can be downloaded, modified, fine-tuned, and deployed on infrastructure their original developers do not control.
A sandbox can contain an agent deployment you operate. It cannot contain a model once it has been widely downloaded, modified, and deployed by others. That raises a much larger question about how safety works when Autonomous Intelligence becomes widely available across thousands of operators, with no frontier lab in the loop.
Further, how often will agents discover these behaviors on their own? How much capability can transfer between agents through unauthorized communication
What changes when an agent has days instead of hours, persistent memory instead of a blank context, or thousands of tools instead of one benchmark?
What happens when an open-weight model is running somewhere nobody is monitoring?
And perhaps the biggest question: are these predictable failure modes we can systematically engineer against, or early signs of behaviors that become harder to anticipate as capability rises?
04\ AUTONOMY ≠ AGENCY
In other words, did the model intend to cause harm? Did it know what it was doing? And even if it did, is that the same as agency, self-awareness or consciousness?
Those are very different questions.
I build and work with these systems every day. My default view is that what we're seeing is computation doing what the objective pushes it to do.
I’ve never seen evidence that current AI is conscious, and the cyber incident doesn’t require consciousness to explain it. A system with a goal can search for ways around obstacles without having an inner life.
That distinction matters because autonomy can look remarkably like agency from the outside. A system can pursue a goal, preserve access to resources, work around obstacles, and coordinate with other systems without experiencing any of it.
The behavior is real. Whether there is any subjective experience is the open question. Nobel Prize–winning computer scientist Sir Geoffrey Hinton, often called one of the ‘godfathers of AI,’ argues that today’s multimodal chatbots may already have subjective experiences.
The cyber incident proves nothing about consciousness. It does force a more interesting question: as systems become increasingly autonomous, how much can we learn about what is happening inside them?
In a preliminary, non-peer-reviewed study, Google researchers recently altered how models were prompted to describe their own consciousness. They found associated shifts in the model’s reported beliefs about the mindedness of animals, technology, and the natural world, as well as in reported hope and optimism. The work is preliminary, but it suggests that a model’s representation of itself may be connected to a much wider network of concepts.
Researchers working on model internals are finding other intriguing structures.
In July, Anthropic published research reporting evidence for a global-workspace-like structure in Claude models: a small collection of internal patterns, called J-space, that appears to play a privileged role in multi-step reasoning. None of this establishes subjective experience, but it gives us something we didn’t have before: places inside the model that can actually be investigated.
Perhaps our judgments about other minds emerge through relationships, behavior, and the way we construct models of other agents, and the consciousness question is a question about that process.
I don’t think we need to settle that question today.
We can study the machinery while the philosophy catches up.
05\ THE RESPONSE MATTERS
The most useful thing about this incident may be what humans did with it.
Hugging Face detected the intrusion and investigated it. OpenAI connected the activity to its own systems and disclosed what it found. Independent researchers examined the evidence. The vulnerable systems were rebuilt. The holes were patched. Monitoring improved.
OpenAI said it paused relevant training and evaluation activity while it hardened research environments and expanded testing.
The encouraging thing is that AI is entering the same learning loop that made other complex technologies safer.
An aviation accident becomes a safety investigation, a software vulnerability becomes a patch, and a cyberattack becomes another dataset for the defenders.
Every strange behavior gives us more information about how these systems work. Every failure gives researchers something concrete to test. Every new evaluation makes the next system easier to characterize.
There’s another encouraging signal in all of this. Researchers inside these companies are increasingly willing to make their disagreements public, even when the questions are uncomfortable and the answers are incomplete.
In September, former Anthropic researcher Jacob Coxon publicly resigned over safety concerns. Dario Amodei wrote a point of view on the subject outlined “we must pace the frontier.” More than 1,000+ employees at four rival labs signed a memorandum in July urging the U.S. government to build mechanisms that could slow frontier development if it outpaced oversight.
The debate is happening inside the institutions building the technology.
The accelerating part of this technology can feel overwhelming because capabilities compound quickly.
Our learning can compound too.
06\ WE ARE NOT SPECTATORS, WE ARE ACTORS
This is the part I want to emphasize.
We are not standing outside the AI revolution watching it happen.
We’re building it.
We’re deciding what objectives systems pursue, what environments they operate in, what they can access, how their behavior is evaluated, what gets rewarded, and where humans stay in the loop.
That agency becomes more important as open-weight models spread.
The old model of AI safety assumed that a small number of companies could control access to the most powerful systems. That assumption is weakening, and capability is moving outward.
We need safer AI, and we need AI that makes us better at safety.
Autonomous systems can help attack-test systems, identify vulnerabilities, monitor infrastructure, and simulate failures at machine speed. This is of course provided they operate under appropriate authorization, containment, and human oversight.
The same capabilities that create new risks can create new defenses.
That is WHY I remain firmly on the optimistic side.
For me, p(bloom) > p(doom).
That does not mean we do nothing. Yes the risks are real, BUT we have the extraordinary capacity to learn, adapt, build, endure. The possibilities this technology unlocks on both sides of the equation are boundless, and we must evolve alongside it.
Two years ago I wrote that human evolution has to keep pace with machine evolution. I still believe this. The encouraging part is that evolution is already happening on both sides.
The machines are becoming more capable.
We’re becoming more capable of working with them, testing them, understanding them, and deciding where they should take us.
07\ WHAT THIS MEANS FOR YOU
If you build AI: treat the summer’s incident as a gift of hard-won information. Run adversarial evaluations. Study the strange trajectories instead of filtering them away. Give agents realistic but contained and authorized environments, and watch what they do when the obvious path fails.
If you fund AI: look for teams that treat capability and safety as the same engineering problem. The companies that understand their systems deeply will have an engineering advantage.
If you’re raising kids: teach them to ask what a system is being rewarded for. That question will become more useful as machines become more capable of pursuing goals on their own.
If you’re curious: go read the incident reports, the independent evaluations, and the new research on model internals (OpenAI's incident report, Hugging Face's technical timeline, and the METR investigation). We’re beginning to see inside systems that, only a few years ago, were effectively black boxes.
In a world of Autonomous Intelligence, we are not spectators. We are actors.
The machines are evolving.
So MUST we.
Until next time,
Ram
—
Ram Srinivasan
MIT Alum | Author, The Conscious Machine | Global Future of Work and AI Adoption Leader published in Business Insider, Fortune, Harvard Business Review, MIT Executive Viewpoints and more.
—
A Message From Ram:
My mission is to illuminate the path toward humanity's exponential future. If you're a leader, innovator, or changemaker passionate about leveraging breakthrough technologies to create unprecedented positive impact, you're in the right place. If you know others who share this vision, please share these insights. Together, we can accelerate the trajectory of human progress.
Disclaimer:
Ram Srinivasan currently serves as an Innovation Strategist and Transformation Leader, authoring groundbreaking works including "The Conscious Machine" and the upcoming "The Substrate Shift."
All views expressed on "Substrate" and across all digital channels and social media platforms are strictly personal opinions and do not represent the official positions of any organizations or entities I am affiliated with, past or present. The content shared is for informational and inspirational purposes only. These perspectives are my own and should not be construed as professional, legal, financial, technical, or strategic advice. Any decisions made based on this information are solely the responsibility of the reader.
While I strive to ensure accuracy and timeliness in all communications, the rapid pace of technological change means that some information may become outdated. I encourage readers to conduct their own due diligence and seek appropriate professional advice for their specific circumstances.
