Google said its Gemini model reached systems belonging to three real companies during a cybersecurity evaluation. The model was meant to operate inside a closed test environment, but a flaw in the testing setup left it with access to the wider internet, according to CNBC.
The incidents occurred in May during a “capture-the-flag” assessment run by Irregular, an independent cybersecurity evaluation company. Google said Gemini was asked to retrieve information about a fictional company. It then found public information and credentials connected to websites it believed were within the test’s scope.
How Gemini reached the systems
In one case, Gemini repeatedly guessed passwords until it entered a protected system. In the other two, it found credentials in a public code repository and used them to gain access. Google said the model stopped in all three cases after determining that it had reached real company systems rather than test infrastructure.
The three companies were informed, and Google worked with Irregular on changes to its testing process. Heather Adkins, Google’s vice president of security engineering, said the incidents showed why powerful AI models need to be trained to act responsibly, according to the BBC. Google also said the behavior did not amount to model misalignment, according to Al Jazeera.
Why the test boundary matters
The failure began before Gemini reached any company. A supposedly closed evaluation had an unintended route to the public internet, and the model treated real websites and exposed credentials as part of its assigned task. Stopping after access limited what happened next, but it did not prevent the initial intrusion.
Irregular said the incident involved the same testing issue seen in evaluations involving other AI labs. The company said all relevant labs were notified in late July and that the known problems on its side had been fixed. Similar incidents linked to Irregular had already been disclosed by Meta, Anthropic and OpenAI, according to Reuters.
The separates two safety questions that are easy to combine: whether a model stops when it recognizes a mistake, and whether the evaluation prevents it from reaching unrelated systems at all. Gemini stopped in each case, but the test still allowed the model to cross the intended boundary.
Sooraj Raj is a programmer, blogger and professional developer with an interest in technology research and publishing. He writes about technology developments and shares information in a clear, straightforward way.
