Microsoft discovers critical security flaw in software that prevents creation of deadly toxins or pathogens

A team of Microsoft researchers have used artificial intelligence ( AI ) to discover a "zero-day" vulnerability in the biosecurity screening software . This software is designed to prevent the misuse of DNA to create deadly toxins or pathogens. The discovery, which was led by Microsoft’s chief scientist, Eric Horvitz , shows how AI can bypass existing protections in a way previously unknown to biosecurity defenders. The findings, published in the journal Science, stated that the researchers focused on generative AI algorithms, the same types of programs that fuel drug discovery at major startups. This software has "dual-use" potential to generate both beneficial and harmful molecules. Microsoft initiated a "red-teaming" test in 2023 to assess whether "adversarial AI protein design" could aid bioterrorists in manufacturing hazardous proteins.

To execute the attack, Microsoft used several generative protein models, including its own EvoDiff, to subtly redesign known toxins. This manipulation was predicted to preserve the deadly function while allowing the DNA sequence to bypass the biosecurity screening software used by commercial vendors.


Microsoft warns how this flaw can be misused

Microsoft confirmed that the exercise was entirely digital and that no toxic proteins were ever produced. Before publishing the results, the company alerted the US government and software makers, who have since patched their systems.


However, the fix is not complete. Adam Clore , director of technology R&D at Integrated DNA Technologies and a coauthor on the report, warned that some AI-designed molecules can still escape detection, stating,


In a statement, Adam Clore, director of technology R&D at Integrated DNA Technologies, a large manufacturer of DNA, who is also a coauthor on the Microsoft report, said: “ The patch is incomplete, and the state of the art is changing. But this isn’t a one-and-done thing. It’s the start of even more testing. We’re in something of an arms race.”


To prevent misuse of the research, Microsoft did not disclose some of its code or reveal which toxic proteins (such as ricin or infectious prions) the AI was tasked with redesigning. However, some experts doubt that focusing solely on commercial DNA synthesis is the best defence.