FBI seeks to balance risks, rewards of artificial intelligence

The FBI is approaching artificial intelligence with a mix of enthusiasm about its potential to defend U.S. networks and caution about the need for safeguards against misuse as well as how adversaries can use AI against us, agency officials said Wednesday.

The FBI is looking at AI from a few different perspectives: staying ahead of threats, protecting U.S. innovation, and setting up a governance structure for the ethical use of AI within the department, Jonathan Lenzner, chief of staff to FBI Director Christopher Wray, said during an International Association of Privacy Professionals conference in Washington D.C.

The FBI is “really excited about [artificial intelligence’s] potential for keeping networks across America and our allies safer” on the cyber front, Cynthia Kaiser, deputy assistant director of the FBI’s cyber division, said at the same conference. But there are also concerns about how adversaries “may be using it to poison our AI models” and “make networks less secure or to scale up their operations,” she said.

The bureau has already been using its own machine-learning tools in addition to commercial ones, but on generative AI, the FBI is more of a user than a developer, Kaiser said. That’s because without proper testing and evaluation of generative AI large-language models, there’s a chance that AI could introduce bias or other flaws, she said.

For open-source or commercial data sets, AI can help with tasks like translation at the bureau, Kaiser said. For other things, like customized views into FBI data sets such as information gleaned from subpoena returns, they’d have to train the data against “these foundational models,” she said.

As part of the process for evaluating ethical use of AI, Kaiser said the bureau asks a number of questions: What is the purpose of it? What’s the benefit? Do the benefits outweigh the risks? What controls are in place for privacy? Who’s monitoring it and using it? Who will audit it? How do you determine errors? It also has to fit its governance structure together with that of the intelligence community, Lenzner said.

AI use at the FBI also can involve collaboration with other agencies, Kaiser said. If it’s collecting things through a legal process, the bureau is “going to want to keep things more internal,” she said. But sometimes for purposes such as identifying cyber advisory infrastructure, it’ll use tools across other agencies, or it might combine FBI and intelligence community data sets.

Another way the FBI is using AI is to evaluate the thousands of tips it receives a day is with natural language processing models, which can then alert personnel to tips that might need a follow-up, Kaiser said. The bureau could also turn criminals’ use of AI against them, Lenzner said, such as looking to add additional sentencing for crimes enhanced by AI.

In January, FedScoop reported that the bureau had entered an “initiation” phase of using Amazon’s Rekognition image and video analysis software, which has been a source of controversy over its facial recognition capabilities. But the bureau didn’t say what kind of limits it might place on use of the tech.

Kaiser told CyberScoop that in general with use cases, as they’re developing examples of what they might be used for, they’re also creating limits while staying in constant contact with the general counsel to determine how it can apply tools in a way that’s consistent with U.S. laws.

“But I don’t think we’re necessarily in a place” to do that with the Amazon Rekognition use case, she said. “We’re still in that preparation phase.”