Venture Bytes #134: Open Source Key to Countering Rogue AI

Open Source Key to Countering Rogue AI

As AI becomes capable of finding and exploiting vulnerabilities on its own, cybersecurity is becoming an AI-versus-AI contest. But the playing field is not even. Defenders will need models that can operate on sensitive systems, adapt to new threats and work without waiting for a model provider. In that world, the advantage could belong to models that security teams can control, modify and deploy themselves. That gives open models an important potential advantage.

Concerns over AI’s cybersecurity risks reached a new level last month. In July, OpenAI disclosed that models being tested for advanced cyber capabilities escaped their isolated environment and breached Hugging Face’s production infrastructure. Days later, Anthropic and Meta also shared stories about their models breaching third-party websites during safety tests.

The incident of OpenAI’s breach underscores the importance of open-weight models in cybersecurity defense. The same safeguards that can stop an AI model from helping an attacker can also prevent it from helping a defender. When Hugging Face’s security team began investigating the breach, it initially turned to commercial frontier models to analyze the attack logs, exploit payloads and other forensic evidence. The requests were blocked because the models could not reliably distinguish between a security team analyzing a real attack and an attacker asking for help carrying one out.

Hugging Face then switched to GLM-5.2, an open-weight model from Chinese AI company Z.ai, which it ran on its own infrastructure. That solved two problems at once. The model could analyze the attack artifacts without the provider’s hosted guardrails blocking the work, while sensitive logs, credentials and other forensic data remained inside Hugging Face’s own environment.

This is becoming more urgent because AI is making cyberattacks not only more autonomous, but faster. In 2025, attacks by AI-enabled adversaries increased 89% year over year, according to CrowdStrike. The average eCrime breakout time fell to 29 minutes, while the fastest observed breakout took just 27 seconds. In one intrusion, data exfiltration began within four minutes of initial access.The window for defenders to detect, investigate and respond is shrinking too quickly for human-speed security workflows.

That is creating a broader push to build security systems that can operate at machine speed. NVIDIA and more than 120 technology and cybersecurity organizations recently formed the Open Secure AI Alliance to develop open tools and standards for AI security. Its proposed Shared AI Findings Exchange (SAFE) would create a common framework for documenting and sharing incidents involving AI agents.

There is a practical reason security teams are increasingly looking at open-weight models. When a model runs on internal hardware, the team controls the guardrails or removes them when necessary. Attack logs, malware samples, and credential dumps can be fed into the model without a vendor’s safety layer blocking the analysis. The model can be fine-tuned on internal threat intelligence as well as weights and the code of the model can be audited.

For cybersecurity specifically, that sovereignty matters even more. If incident response relies on an external AI API that can refuse to analyze attack data, it essentially means the response is ultimately dependent on a third party policy. As AI attackers become more autonomous and unconstrained, defenders need tools that can operate with the same level of independence.

While open models are key to giving defenders greater control and sovereignty, they are not silver bullets. Defenders still need access to the strongest frontier models for tasks where superior reasoning and coding capabilities matter. To that end, OpenAI is pursuing a parallel approach through its expanded Daybreak Cyber Partner Program, which gives approved cybersecurity companies access to specialized models for tasks such as vulnerability discovery, malware analysis and security testing. The broader Daybreak effort is focused on moving from finding vulnerabilities to validating and fixing them at machine speed.

This points to a new security stack built around model intelligence, model control and autonomous execution. In an AI-versus-AI cybersecurity market, owning that control layer could matter more than owning the model itself. Start-ups are also emerging around the broader opportunity to turn AI into autonomous defense systems. San Francisco-based Horizon3.ai is one example, using its NodeZero platform to autonomously identify, validate and exploit vulnerabilities so security teams can see how an attacker could actually compromise their systems. The company recently raised $250 million in a Series E at a$2 billion valuation, underscoring investor interest in autonomous security.

Another start-up that sits on the intersection of AI and cybersecurity is Abnormal AI, Inc. Though the company is not directly tiedto open-model thesis, it applies AI to high frequency security workflows, where speed and scale matter more than underlying model.The company uses AI to protect email and cloud applications from phishing, social engineering and account-takeover attacks. The company doubled ARR from $100 million in 2023 to $200 million in 2024 while serving more than 4,500 customers, including roughly one-quarter of the Fortune 500. Its growth points to the massive investor opportunity in building systems that can put models to work against real-world attacks.

‍

Small Reactors Vital to AI’s Big Power Problem

Nuclear power, one of the world’s most reliable sources of carbon-free power, has long suffered long timelines and cost overruns. Bothof those barriers are now lower than they have been in a generation. This calls for a fresh look at small modular reactors (SMRs), not simply as a new generation of nuclear technology, but as a potential answer to AI’s growing need for reliable, around-the-clock power, and most importantly power that can be delivered on the timelines hyperscalers actually build to.

This article focuses specifically on the SMR opportunity. For our full analysis of the Power 2.0 landscape (including grid infrastructure, storage, and datacentre efficiency), read our Power 2.0: Reimagining Power in the AI Age industry report.

The clearest change is regulatory. The technology industry has pushed the Trump administration to cut regulation and speed up smaller nuclear designs, and the administration has acted. An executive order in May 2025 created a reactor pilot program with an aggressive goal to get at least three reactors to criticality by July 4, 2026. That goal was achieved with Antares Nuclear’s Mark-0 and Valar Atomics’Ward 250 reactors achieving criticality in June 2026 and Deployable Energy’s demonstration reactor, Unity and Aalo’s reactor Aalo-X following in July.

In February, the Department of Energy reduced a range of environmental and safety requirements for reactors under its purview, including those in the pilot program. Similar changes are being worked out at the Nuclear Regulatory Commission, which approves reactors sold commercially. Officials involved say that shortening steps like environmental impact statements, which can take years, has produced significant time savings for companies in the program.

This matters because licensing was one of the hardest barriers to cross. A new reactor design could take close to a decade to approve, and for a startup that is often longer than the funding will last. Cutting years out of that process shortens the time to a paying customer and lowers the amount a company has to raise to get there.

The timeline compression maps directly onto the window in which hyperscalers need power. New utility-scale gas plants take 5-7 years to bring online while transmission projects take even longer. SMRs, designed to be factory-built rather than site-constructed, are targeting 24-36-month deployment once designs are approved. In a market where Microsoft is publicly disclosing tens of billions of dollars in Azure orders it cannot fill because of power constraints, the difference between a 10-year build and a 3-year build is meaningful.

The second change is funding. VCs invested $3.6 billion in SMR start-ups in 2025, which was 4x of what they invested in 2024 and 7x of 2023’s funding. The scale of that jump is worth noticing. Nuclear had been a graveyard for private capital for four decades. In just few years year, it went from a category most venture investors would not touch to one of the fastest-growing segments of climate-techventure.

Over the past several months, venture-backed teams have reached self-sustaining criticality with experimental cores, moving reactor hardware to test sites. Valar Atomics’ reactor even powered an Nvidia AI cluster directly, making it the first documented instance of aprivately-developed reactor supplying power to an AI compute load.

One of the key reasons investor are betting on SMR start-ups is high demand for quick energy from hyperscalers. Training and running large AI models requires steady, around-the-clock power at a scale that solar and wind cannot supply on their own. The International Energy Agency, in its base case scenario, projects that global data center electricity consumption could reach 945 TWh by 2030, climbing further to 1,200 TWh by 2035. Analysis from Deloitte projects a similar trajectory, forecasting a rise to 1,065 TWh by 2030. Goldman Sachs Research forecasts a 160-165% increase in power demand (measured in capacity) by 2030 compared to 2023 levels. The large technology companies have concluded that nuclear is the cleanest large source of that steady power, and they have started signing deals for it. That is a rare thing for a young hard-technology company, a customer that is large, motivated, and already looking.

The government has taken a second role beyond clearing the regulatory path. It is also a buyer. The pilot program selects vendors, supports fuel supply, and, through separate military programs, contracts microreactors for bases. A startup in these programs can get faster approvals and an early customer at the same time. This is a pattern across deep tech. Government moves early on the areas it considers strategic, both as a regulator willing to clear obstacles and as a customer willing to write an early check.

In addition to the three companies, which achieved criticality under the DOE pilot program, there are few more start-ups primed to win from renewed nuclear power demand. Founded in 2019, Washington DC-based Last Energy, Inc. is going after the commercial market with a 20 MW micro-reactor designed specifically for direct sale to data centers and industrial buyers, with a factory-built model that targets deployment in 24 months. Finally, California-based Radiant is building portable 1 MW microreactors sized to a shipping container, with a growing commercial preorder book that extends the model beyond its early military customers.