How AI could make it harder for governments to use hacking tools

Earlier in August, cryptography professor Matthew Green wrote a controversial thread on X and a longer blog post that went viral within the cybersecurity community.

Green, who has long been a close observer of the debate around the use of hacking tools by governments to fight crime and the need for strong encryption to protect the privacy of innocent people, posited a provocative thought: What if AI makes bugs so scarce that law enforcement and intelligence agencies are unable to lawfully hack criminals anymore?

“I’m concerned that AI is going to make software much too secure,” Green wrote, warning that the U.S. government may lose access to security flaws to hack into targets they need to surveil as companies patch an unprecedented volume of bugs.

Law enforcement have long claimed that encryption made it difficult to catch criminals and terrorists. The concept of “going dark” was popularized in 2014 at a time when then-FBI director James Comey warned that encryption could hamper authorities from being able to listen in on conversations or access data on devices.

Around this time, apps like Signal, WhatsApp, and Apple’s iMessage rolled out end-to-end encryption to the masses, making traditional real-time wiretapping of calls and text messages almost impossible. Tech giants like Apple also began making data on their devices encrypted by default, making it harder to break into iPhones protected by a strong PIN code or passphrase. 

Since then, authorities have still been able to catch criminals — including by hacking into their devices — and innocent people have been able to enjoy a good level of privacy thanks to encryption. In part, as Green explains, that is because of “an uneasy kind of truce.” That is, instead of incorporating backdoors into devices to help authorities get the data, governments have instead invested money into buying hacking tools and spyware that can subvert the security of devices and their owners.

For Green, that truce is about to be disrupted by AI, because, as proponents promise and some early data suggests, LLMs are becoming better and faster at finding security vulnerabilities at scale. That, in theory, suggests we will get to a point where companies can make their software and systems significantly less bug-ridden — and prone to attacks. 

The end result, per Green, is that governments could ask for backdoors again, making everyone’s devices less secure by design.

A gold rush of bugs

We asked several people to chime in on Green’s argument, from privacy and cybersecurity experts to hackers who have experience developing hacking tools for governments. Some agree with Green, some disagree, and some see it both ways.

Luna Tong, a researcher who has previously worked at two prominent companies that search for bugs and develop exploits to help governments break into systems, agreed with Green, saying that there is a “gold rush of bugs right now but it’s a temporary phenomenon and bugs will get scarce again soon.”

Another researcher, who has more than a decade of experience working at offensive security firms, said that he is worried AI could make human security researchers obsolete because it will be much harder to find bugs, and that defenders will eventually have the edge over offensive researchers. The person asked not to be named so that they could speak more freely.

“It’s clear that no state will throw away the possibility of surveillance,” said Paolo Stagno, the chief technology officer at Crowdfense, a well-known company that develops, acquires, and sells unknown vulnerabilities — also known as zero-days — to governments. Stagno explained that the current process of requiring governments to exploit security flaws to break into devices is the “most democratic system we have,” but that the status quo may not last if bugs become too hard to find. 

Three other people who currently work in the offensive cybersecurity industry, and one who used to, disagreed. Their arguments boil down to: Easy bugs will be easier to find; more complex bugs that are generally more valuable and useful for governments will not go away; and, AI can actively assist the researchers who sell bugs to government authorities. 

Hamid Kashfi, who is the founder of offensive security firm DarkCell and also works at the AI cybersecurity startup Xbow, said that “for every AI found and reported bug out there, there are probably 20 that are not reported.” Kashfi explained that researchers who do not want to report bugs to vendors can still find complex and valuable bugs. 

Two of the researchers who currently make a living looking for bugs for zero-day firms told TechCrunch that they were less concerned about the rise of AI than they were about a slew of new security protections in modern devices that make them more difficult to hack. 

Eva Galperin, the director of cybersecurity at the digital rights Electronic Frontier Foundation and an expert on government spyware, said that offense has the advantage today due to a combination of AI being highly capable of finding bugs, and an increase in the number of vulnerabilities introduced by “vibe-code” developing with AI tools. 

On the other hand, Galperin argued that finding more bugs doesn’t necessarily mean more bugs will be patched fast enough, or even at all, given that patching can be a complex process. Galperin said that there will still be a renewed push for backdoors at some point because authoritarian regimes always want “exceptional access.”

Katie Moussouris, who has helped companies both big and small deal with reported bugs and patch them for decades, said that, “we have some distance to go before the latest phones and laptops are completely bug free.”

“There will be some point at which finding bugs will be much harder and that may trigger these pressures to build in backdoors,” said Moussouris, the founder and CEO of Luta Security. 

“I think we have at least until after the next presidential election before the intelligence community is materially hampered enough to push for backdoors in a serious way,” said Moussouris.