‘GridEx’ offers stiff security test for an industry that welcomes the challenge

Every two years, power-grid authorities throw the kitchen sink of digital and physical mayhem at electric utilities and government organizations across North America.

It is one of the biggest tests of the utilities’ ability to withstand wave upon wave of hypothetical attacks — and they are not necessarily supposed to pass the test.

The GridEx simulation, which begins Wednesday, is “purposely designed to overwhelm even the most prepared organizations” so they can improve their resiliency, said Matt Duncan an official at the North American Electric Reliability Corp., which runs the drill.

Exercise participants won’t need any reminders that, in the last four years, malicious hackers have cut power for hundreds of thousands of people in Ukraine and caused a petrochemical plant to shut down in Saudi Arabia. GridEx is one way that U.S. critical-infrastructure companies work to prevent such disruptive attacks from hitting them.

Participants, which will also include natural gas companies and telecom firms, plug into the exercise from their workstations across North America. They choose from an array of fictional scenarios, depending on their skill level. That can mean actual hands on keyboards defending simulated attacks, or staying a step removed by discussing different threats.

“We really want to highlight the growing interdependence of critical infrastructure” from various sectors, said Duncan, who works at NERC’s Electricity Information Sharing and Analysis Center (E-ISAC).

Participants will have to deal with simulated fuel shortages, compromised customer payment systems, and the specter of copy-cat attacks following those of a state-sponsored adversary, Duncan told CyberScoop.

However long-shot and doomsday-like the training scenarios are, they are grounded in real-world threats. In April 2018, hackers hit the customer payment systems of U.S. natural gas companies. And the power-sector supply chain, which will be a focus of the exercise, has been targeted by the group behind the dangerous Trisis malware.

Supply-chain security will be front and center

This year’s GridEx will emphasize supply-chain risks like never before, feature a greater number of small power companies than usual, and for the first time test an emergency order from the secretary of Energy to secure the grid.

Six “supply chain vendors” attended the last GridEx, in 2017. But to the dismay of organizers, the utilities didn’t really interact with the vendors during the exercise, according to a post-exercise report.

This time should be different.

Adrienne Lotto, senior director of enterprise resilience at the New York Power Authority, which will participate in GridEx, said the supply-chain scenario could offer important lessons for participants.

“It will be interesting to see how that interaction plays out and what sort of resiliency and operational changes that would have to be made as a result,” said Lotto, who was previously a deputy assistant secretary at the Department of Energy.

The second day of GridEx features a tabletop drill with industry executives and federal officials, including the Department of Homeland Security and the FBI. For the first time at GridEx, participants will test how an emergency order to secure the power grid might unfold. Under a 2015 law, the president can declare a grid security emergency, leading to an order for utilities to take drastic action to fend off attackers and keep the lights on.

Open minds, closed ports

Utility executives will enter GridEx with a more mature security mindset than in 2011, when the exercise began. That’s because more and more companies have been willing to let third-party, white-hat hackers rigorously attack their control-system networks in order to make them more secure. Independent cybersecurity tests are nothing new to the industry, but the tests have grown more sophisticated as malicious hackers have, too.

The shift is illustrated by a request that Shawn Duffy, a white-hat hacker, got some five years ago from an oil and gas company: Break into our control-system environment and tell us what physical effects you might be able to cause.

Duffy’s team spent months plotting and gaining access to the oil and gas company’s operational-technology network. Moving to a lab environment, the white hats showed how they might be able cause pressure to build in the facility’s valves, potentially leading to an explosion.

At the time, it was the furthest a client had allowed Duffy’s team at FusionX, which Accenture has since acquired, to go on an industrial network. But in the years since that test, Duffy says he has carried out that kind of rigorous red-teaming at other client facilities.

“I think there’s…more awareness that these types of attacks are possible” and are capable of happening in the real world, Duffy told CyberScoop.

Security specialists have also given energy facilities a wider menu of options to choose from in terms of security tests, from full-on red teaming to assessments that outline how an adversary might get into a network, to penetration tests, or pen-tests, of specific equipment.

Leslie Adams, senior industrial penetration tester at cybersecurity company Dragos, said it doesn’t take digital wizardry to do a good test. “A lot of our [pen-testing] toolsets have not changed at all, and that aligns with the tactics, techniques, and procedures the adversaries are using,” Adams told CyberScoop.

Experience with all of those tools will help participants get more out of GridEx this week.

“Back in 2011, I think there was a lot of apprehension in participating in a cybersecurity exercise with your regulator,” said Brian Harrell, who stood up the first GridEx when he was a NERC official.

“Today, I think a lot of that apprehension has subsided,” said Harrell, who is now assistant director for infrastructure security at DHS’s Cybersecurity and Infrastructure Security Agency. “I think NERC has really demonstrated that they do this exercise to collectively” improve security.