NSA’s Cybersecurity Directorate is still figuring out how to measure success
Since the National Security Agency established a new directorate focused on cybersecurity, the organization once known as “No Such Agency” has engaged in some behavior that would have seemed revolutionary a decade ago: publicly sharing information about several large-scale hacking threats, including Russian hacking incidents, as well as information about a critical Microsoft vulnerability.
How successful the agency considers that behavior is still something it’s examining.
The NSA’s Cybersecurity Directorate, which was established last October in part to share more threat intelligence with the public and the private sector, has been examining the impact of its Cybersecurity Advisories in a variety of ways, the NSA’s Executive Director, Wendy Noble, said during a virtual Billington CyberSecurity Summit Wednesday.
“The more important thing to track is how [CSD information gets] used, the operational outcome,” Noble said. “We are working to develop those metrics to make sure we understand the value proposition … how it benefits government, how it benefits industry, and how it benefits our allies.”
The Cybersecurity Directorate is also meant to support warfighters, enhance critical network defense, and work on the keys, codes, and cryptographic solutions that underlie secure communications for the government. Noble admitted the directorate is still solidifying how it measures performance when it comes to supporting the U.S. government and allies on this side of the directorate’s security and assurance mission.
But Noble said Wednesday the public-facing announcements are also being measured, particularly through press outreach and social media chatter.
Last October, it was not clear if the directorate had established standards to measure its impact. At the time, directorate’s lead official, Anne Neuberger, said her team would be asking government partners if the group’s work had made a difference in their security posture.
Noble told CyberScoop that the Cybersecurity Directorate is currently examining mitigation adoption in the government and feedback from the information security sector to determine if it’s meeting goalposts.
“We analyze the impact of NSA Cybersecurity products by looking at several factors to include mitigation adoption among customers and broader analysis of feedback among the cybersecurity community. The goal is always to ensure we are providing timely, unique and actionable guidance,” Noble told CyberScoop in a statement.
Some of the directorate’s programs focused on enhancing government security have trickled out into the public eye over the last year, but they are not yet completed, making impact somewhat difficult to ascertain. For instance, the directorate announced in June its effort to pilot a secure Domain Name System to reduce malware-based threats targeting contractors working on DOD weapons technologies.
The NSA has also been working to share information about threats early with network defenders. When the Cybersecurity Directorate announced it had uncovered a critical Microsoft crypto API flaw in January, the NSA said it had provided advance notice to critical network owners in an effort to ensure they could apply the patch with an adequate amount of notice, for instance.
The NSA’s efforts to share information on election threats with the Department of Homeland Security’s cybersecurity agency has helped improve broader information-sharing efforts, according to Chris Krebs, the Director of the Cybersecurity and Infrastructure Security Agency.
“Are things perfect? Absolutely not,” Krebs said at CyberScoop’s San Francisco CyberTalks in February. “This is going to be a lifetime — a generational engagement on election security.”
