Here’s what Anthropic found when it turned Mythos loose on encryption algorithms

Anthropic researchers used Claude Mythos Preview to find new weaknesses in two cryptographic methods, the company said Tuesday, including one that is being considered by the National Institute of Standards and Technology for both traditional and quantum computing.  

In a blog post detailing the work, the frontier AI company called it a “substantial” research advancement, but also emphasized that neither flaw affects software now in use.

“The attacks described in these two papers are the strongest attacks we have found to date,” the company wrote in the post. 

One of the weaknesses found was in HAWK, a digital signature scheme under review by the NIST as part of a search for encryption methods that could survive attacks from quantum computers. Working with a human researcher, the AI system found a mathematical shortcut, known as a nontrivial automorphism, in the lattice structure (a complex mathematical grid underpinning its security) HAWK relies on.

The discovered weakness cuts HAWK’s effective key strength in half, meaning key sizes would need to double to keep the same level of security. Anthropic said that change would erase much of what made HAWK an appealing candidate in the first place.

Ellen Boehm, senior vice president of strategy and AI innovation at Keyfactor, a digital identity and cryptography management provider, told CyberScoop that research like Anthropic’s proves that the NIST PQC evaluation process is working. 

She also said the research “elevates the importance for organizations to have visibility of where cryptography sits inside their enterprise, what business systems and processes it’s connected to, and the need for PQC readiness, if they haven’t already built a plan.” 

The other flaw was found in a weakened version of the Advanced Encryption Standard, or AES, the cipher NIST adopted in 2001 and the most widely used method for scrambling data in transit. Working largely on its own, Mythos invented a mathematical shortcut dubbed the “Möbius Bridge.” While real-world encryption scrambles data through 10 sequential layers, or “rounds,” researchers regularly study a simplified seven-round test version to measure security margins. In previous theoretical attacks, codebreakers had to check 256 separate values against a memory table, but Mythos created a shortcut that eliminated that lookup process entirely.

Combined with other optimizations, this discovery made the strongest known theoretical attack against seven-round AES 200 to 800 times faster. The attack is purely theoretical: It requires an impossible amount of target data — over 400 octillion messages — and cannot touch the full 10-round encryption protecting everyday software. Additionally, Anthropic pointed out that real-world systems remain completely safe.

Anthropic said it followed standard disclosure practices, notifying HAWK’s designers in June and coordinating public release with a NIST mailing list, and briefing government and industry partners beforehand. It also worked with researchers at ETH Zurich, Tel Aviv University and the University of Haifa to build a shared testing tool, called CryptanalysisBench, meant to let other researchers measure how AI systems perform against a range of ciphers.

The findings come as frontier AI models are being deployed by cybersecurity researchers in order to find vulnerabilities in all kinds of software. In June, intelligence agencies in the Five Eyes alliance warned that advanced AI models capable of wreaking havoc in the cyber domain are “months away.” However, a recent report found that despite the avalanche of bugs being unearthed, the threat level across the internet has not materially changed. 

Anthropic said it expects the same AI capabilities eventually to be applied to systems already in wide use, raising a separate question it said it has not yet resolved: how researchers, companies and governments should respond if a language model uncovers a flaw in a cryptographic system that protects critical infrastructure.

“As we develop increasingly powerful cryptanalytic results, it would be prudent to consider how researchers should react if a language model were to discover vulnerabilities in cryptosystems where attacks do have an immediate real-world impact,” the company wrote. “We hope that our work here will help launch these conversations.” 

Boehm said work like Anthropic’s further shows that enterprises should not rest on their laurels with any facet of their security apparatus. 

“AI is becoming a powerful tool for many things, including software quality assurance, code development, and in this case cryptographic analysis,” she told CyberScoop. “As AI tools become more widely and continuously used, it just elevates the need for enterprises to treat their trust infrastructure in an ongoing, operational manner versus thinking of it as a static environment that only changes every few years as new cryptographic algorithms are released.”