This firmware flaw was bad enough, but then researchers looked at the supply chain

When researchers first found critical vulnerabilities in the firmware of certain Lenovo computer servers, it looked like a fairly straightforward issue. The problem, however, involved far more than the Hong Kong-based PC giant.

The vulnerabilities were in the firmware of baseboard management controllers (BMC), the small processors used to remotely manage servers at an organization. The flaws could allow an attacker to run arbitrary code within the BMCs to retain persistent access to a computer system, or to “brick” the BMC entirely, rendering it inoperable.

Those facts alone were cause for concern, but specialists at hardware-security company Eclypsium discovered a bigger story. The firmware in question was actually sourced from another company — Ohio-based Vertiv — and it was present in servers made by at least seven other vendors.

“That’s when we realized just how complex and vulnerable the BMC supply chain is,” said Jesse Michael, principal security researcher at Eclypsium.

The episode highlights how widespread a vulnerability in the supply chain can be, and how fixing it can involve many more companies than initially thought.

“Most hardware vendors do not write their own firmware and instead rely on their supply chain partners,” Eclypsium said in research published Tuesday. “Firmware is quite commonly licensed from a third party and used with little modification, allowing vulnerabilities to extend to many different brands and products.”

Lenovo has issued updates for the remote-command-injection vulnerability, saying the flaw affects older servers. However, the other firmware vulnerability, which involves a failure to cryptographically sign firmware updates, cannot be fixed in older versions of the server, according to Eclypsium. A Lenovo spokesperson could not be reached for comment.

A Vertiv spokesperson told CyberScoop the company was aware of the reports of vulnerabilities in older versions of its products. “[Vertiv is] evaluating the matter and will determine if any actions should be taken,” the spokesperson said. “They will be able to share more information as soon as they complete their evaluation.”

Advanced groups have hacked servers to send malicious updates to users before. Last year, hackers involved in an operation known as ShadowHammer compromised a server belonging to Taiwanese hardware manufacturer ASUS, sending malicious updates that affected an estimated 1 million users.

BMC vulnerabilities are neither a new phenomenon nor confined to certain vendors. Hardware giants like HP Enterprise and Dell have grappled with BMC issues, the Eclypsium researchers pointed out.

Two remedies to this security challenge, Eclypsium said, are for manufacturers to rigorously test the firmware they license, and for organizations installing the firmware to scan it for bugs whenever they accept a new device onto their networks.

UPDATED, 07/18/19, 9:36 a.m., EDT: This story has been updated with a statement from a Vertiv spokesperson.