Trisis investigator says Saudi plant outage could have been prevented

Engineers and others responding to malware that hit a Saudi Arabia petrochemical plant in June 2017 missed a key opportunity to prevent the plant from shutting down a second time in August that year, an investigator of the incident said Tuesday.

“The scope of the initial outage investigation that occurred in June [2017] was insufficient,” Julian Gutmanis, an industrial cybersecurity specialist who responded to the second outage, said Tuesday at the 2019 S4 Conference. “It really was a missed opportunity to identify the attackers and prevent the subsequent outage in August [2017].”

The investigation of the June 2017 outage, which struck on a Saturday evening when the plant was manned by a skeleton crew, included a mechanical and engineering analysis, but not a cybersecurity one, Gutmanis said.  The incident was ruled a malfunction, rather than an attack, and normal operations at the plant were restored. Two months later, the hackers were again rooting around the plant’s operational technology (OT) network, this time hampering six controllers instead of one. Both outages lasted about a week, Gutmanis said, bringing significant costs to the plant in lost production.

The disruption of the Saudi plant was caused by the infamous malware known as Trisis or Triton, which has alarmed industrial control system (ICS) experts since it surfaced because of its potential to cause physical harm. The malware was designed to infiltrate the safety instrumented systems that allow industrial plants to safely shut down, specifically a Triconex model made by energy software giant Schneider Electric.

Gutmanis offered one of the most detailed public accounts of the disturbance at the Saudi plant, including a charge that the vendor of the hacked software failed to communicate information about the attack with incident responders.

Gutmanis told CyberScoop that when he heard about the August outage and he was called to investigate it, the possibility of an explosion occurring at the plant was on his mind. “That was pretty hairy,” he said.

“When I actually found the attack tools on that engineering workstation, that gave me [the] shivers,” the Australian added.

Gutmanis told the S4 audience that his team had to carefully reconstruct what took place at the plant.

“When we went on site, it was really a black box,” Gutmanis said. “We didn’t know the organization, we don’t know the architecture, we didn’t know the people.”

And so investigators interviewed plant employees to gather information, and initially did not rule out the possibility of an insider threat. But signs started pointing to an outsider attacker, and investigators caught a break when the hackers got sloppy.

“The attackers got somewhat complacent because they had been in the environment for a long time, and left tools on the system,” Gutmanis said. Investigators also feared that a follow-on hack could be imminent – something akin to the Shamoon data-wiping attack that occurred in 2012.

On paper, the plant had a secure architecture, Gutmanis said, but that security started to crumble at a poorly configured boundary that separates a plant’s IT and OT networks. As a result, the hackers were able to pivot from one network to the other, he said.

Investigators found malware at the plant unrelated to the Trisis incident that Gutmanis said had been there for years.

A threat group on the move

Schneider Electric executives won praise from the ICS community for presenting what they learned from Trisis at last year’s S4 Conference. However, Gutmanis said the 2018 presentation was the first time he heard about Schneider Electric’s analysis of the attack tools.

The company was initially communicative with incident responders but then failed to inform them of the detection tool for the attack that they were publicly releasing, Gutmanis said.

“[Schneider Electric] promised us on multiple occasions to respond to us and give us information about what was occurring,” Gutmanis told CyberScoop. “However, they didn’t even tell us that they were going to present [at S4 in 2018].”

In a statement to CyberScoop, Schneider Electric said that it sent an engineer to the plant within four hours of support being requested and that Schneider’s on-site experts conducted an analysis of the incident.

“Once they determined the incident to be cybersecurity-related, they turned the investigation over to the end-user, who hired FireEye for attack eviction and site remediation,” Schneider Electric said. “FireEye worked directly with the end-user, and at the end-user’s request, Schneider Electric communicated only with FireEye. At every step, we have cooperated fully with the end-user, FireEye and the U.S. Department of Homeland Security, with coordination from the U.S. Federal Bureau of Investigation. We continue to be open and transparent about the incident to learn from Triton and help the broader goal of worldwide cyberattack prevention.”

As ICS professionals continue to study Trisis, the group behind the malware has been restless. Months after it struck the Saudi petrochemical plant, researchers reported that the group had expanded its operations beyond the Middle East to target U.S. companies. Last October, cybersecurity company FireEye said a Russian government-owned research institute very likely helped build tools used by the hacking group. That attribution didn’t extend to the Trisis malware itself, rather to other hacking tools used by the group.

There has been speculation within the ICS industry that the Saudi petrochemical plant was just a testing ground for future attacks.

“If that’s the case, I’m sure that they’re making more…advanced tools to impact these systems,” Gutmanis told CyberScoop.

UPDATE, 01/16/19, 12:37 a.m. EDT: This story has been updated with a statement from Schneider Electric.