The AI Era Is Creating a Bug Hunting Arms Race

A decade ago, programs to reward researchers for submitting software vulnerability findings were just starting to go mainstream. Vulnerability disclosure and “bug bounty” programs represented a paradigm shift years in the making—moving institutions from hostility and defensiveness about security research findings to acknowledgement that receiving input and releasing fixes was necessary. When Apple finally announced a bug bounty in 2016, the top reward was $200,000. It rose to $1 million in 2019 and $2 million last year. But all of that is about to change again.

As agentic AI models become more adept at both autonomously identifying software vulnerabilities and developing exploits for them—in other words, identifying weaknesses and creating hacking tools—vulnerability disclosure programs are being flooded just as organizations are finding more bugs than ever themselves. This abundance is changing the economics of bug bounties for both institutions soliciting submissions and researchers, some of whom currently make a living or supplement their income with bug hunting. And, crucially, the field is changing in lockstep for attackers, too.

“I’ve probably submitted three times more bugs than I did last year at this time—I would suspect that a company like Google is going to spend two to 10 times as much on bug payouts as they did last year,” says independent security researcher Joseph Thacker, who has developed methods and tools for using AI in his own bug hunting.

Tech giants, he adds, ”can handle that pressure, but most companies can’t. Right now people will be submitting low- and medium-hanging fruit—agents are finding really good bugs. But next year there will be fewer bugs submitted because a lot of that will already have been found, and I think some companies will up their payouts again.”

Thacker and other researchers readily admit, that no one knows exactly how the supply and demand dynamics will play out long term. And depending on how effective AI exploit discovery and automated system scanning is for attackers, developers may start to feel even more pressure to quickly release patches—potentially speeding longstanding and hard-won standards like 90-day disclosure deadlines (set windows between finding bugs and disclosing them publicly that often spur patch releases).

As security researcher Himanshu Anand wrote earlier this month, “The 90 day responsible disclosure window was built for a world where bug finders were rare and exploit development was slow. That world is gone. LLMs have compressed both timelines.”

Crucially, forced accountability by attackers could also motivate improvements in how quickly organizations deploy vulnerability fixes in their systems. Patch proliferation has always been a crucial but complex security challenge given that, without proper testing, installing new software at scale can have unintended consequences, including worst-case scenarios like outages.

The urgency of real-world attacks facilitated by AI seems to be growing, with both sophisticated and less-proficient actors looking to expand their capabilities and cut costs. In findings published earlier this month, for example, Google researchers said that they had observed “prominent cyber crime threat actors” (whom they declined to identify) attempting to exploit a zero-day—or previously unknown—vulnerability that they had developed using AI tools to bypass two-factor authentication on an open source system administration platform. Google quickly notified the developer and they issued a fix for the flaw. But the researchers said that incident was a crucial illustration of the changing bug-hunting landscape.

“We all assumed it was already happening, and this is our first evidence that it is happening,” John Hultquist, Google Threat Intelligence Group chief analyst, says of attackers using AI to discover novel vulnerabilities and create exploits.

“Nation state issues are very serious and very real, but criminal actors still make up the vast majority of incidents that organizations deal with and many of those incidents are quite serious,” Hultquist adds. “Zero-day use by criminal actors has been fairly limited, and the ones that do use them tend to be really successful, so I think we shouldn’t underestimate the impact of more criminals with a zero day in their hands.”

For researchers making money through bug hunting, though, times are changing. The command-line tool Curl ended its bug bounty program (run through third-party service HackerOne) in January after being inundated with low-quality submissions generated by AI.

“We have concluded the hard way that a bug bounty gives people too strong incentives to find and make up ‘problems’ in bad faith that cause overload and abuse,” the group wrote at the time, adding that “we still appreciate and value valid vulnerability reports.”

Last week, Linux creator and lead developer Linus Torvalds wrote that the famed Linux security mailing list has become “almost entirely unmanageable” because of high volume and duplicate AI bug reports.

In April, though, Daniel Stenberg, the founder and lead developer of Curl, said in a LinkedIn post that the quality of submissions had improved. “Over the last few months, we have stopped getting AI slop security reports in the curl project,” he wrote. “Instead we get an ever-increasing amount of really good security reports, almost all done with the help of AI. They're submitted in a never-before seen frequency and put us under serious load.”

And at the end of April, Google announced that it was overhauling its Vulnerability Reward Programs for Chrome and Android and lowering payouts for some classes of bugs, while increasing others.

“As the security research landscape evolves with AI, we're making changes in our programs to ensure we're rewarding the most challenging and impactful vulnerabilities in our products,” the company wrote.

“I think 90th percentile bug hunters with special skills will always be able to have findings and get payouts from big companies," says Jonathan Dunn, a cardiologist who is also a bug bounty hunter. “But even with AI, we also need to heavily incentivize ethical researchers to find stuff on public infrastructure and other critical systems that otherwise may not get enough attention from defenders.”

For now, most organizations seem ready to throw every solution they can think of at the problem (and benefit) of accelerated bug discovery. “This is changing the dynamics of the bug-hunting industry, but it absolutely still requires human time,” says Alex Zenla, chief technology officer of cloud security firm Edera.

Earlier this month, Anthropic launched a HackerOne bug bounty for researchers to submit findings on the company's own systems and Claude AI models. Increasingly, though, some researchers argue that structural defenses are necessary to address accelerating vulnerability discovery. In other words, they're architecting digital solutions for different classes of vulnerabilities that eliminate them or make them significantly less exploitable in practice.

“You can’t patch your way out of this,” says longtime security engineer and researcher Niels Provos. “You need to build infrastructure that makes as many bugs as possible irrelevant.”