SCOVILLE, Idaho — At the Energy Department’s remote "proving grounds" in southeast Idaho’s desert, researchers are testing automated cybersecurity defenses to protect grid systems if a destructive attack comes too fast or in too many places for humans to handle.
A unique replica of a California grid substation has been created here at the Idaho National Laboratory (INL) to allow real-world trials of advanced software intended to find dangerous hidden malware and trigger immediate countermeasures at computer speeds.
The process, called Machine-to-Machine Automated Threat Response (MMATR), aims to improve early warning systems for utilities with advanced threat analysis using technologies not yet commercially available.
The testing of the MMATR defenses here at the INL facilities is part of a high-profile, heavy-lift project called California Energy Systems for the 21st Century (CES-21), directed by the California Public Utilities Commission.
CES-21 enlists the state’s largest public utilities — Pacific Gas and Electric Co., Southern California Edison Co. and San Diego Gas & Electric Co. — and the Lawrence Livermore National Laboratory, along with INL. The goal is to test a wide range of attack scenarios against MMATR defenses tailored to the three utilities’ systems. If it succeeds, the strategy could be widely deployed in the entire U.S. utility sector.
"With attacks becoming more sophisticated and frequent, defending utilities have a very short window to identify an intrusion, notify critical systems, and respond to incoming threats before the issue becomes critical," said Joy Weed, manager of cybersecurity outreach and operations at Southern California Edison, in an email comment.
Behind the project is a grim assumption: that highly sophisticated attackers may tunnel into utility control systems and hide cyber weapons that will await a command to strike. The danger would increase geometrically if versions of the same weapon have infected many power systems and could be activated by a single command.
"The adversaries are very agile and very creative, and our responses are not," said Rita Foster, INL’s cybersecurity strategic adviser. "The responses do not match the threat. They are very staid, controlled, static. So there’s a total mismatch there," said Foster, interviewed in an INL lab space surrounded by sensors and controls from real power grids that are part of the test regime.
"That’s why they can be successful, why they can hide, and why they can do things that we don’t think they can do," she said.
The project developers may have to convince the Energy Department, utilities and state regulators that increased sharing of sophisticated cyber weapons will strengthen defenses but not open unanticipated channels for attack. "If the confidentiality, integrity and availability of an M2M system cannot be ensured, there will be serious problems," said Brian Harrell, vice president of security of AlertEnterprise. "It is very important to construct an effective security framework against various attacks to protect the M2M communication systems."
Backers of the strategy say utilities have no choice but to accelerate defenses.
"Attacks on energy infrastructure can move so swiftly and be so comprehensive — the ones we’re imagining — that humans can’t react in time," said Andrew Bochman, INL’s senior cyber and energy strategist, testifying before Congress this year. "But we can set up our systems so that they can look for and identify certain attack patterns and immediately trigger the remediation action.
"It sounds pretty far out, and it is, but we’ve been making good progress by all accounts," he said.
To succeed, MMATR technology will have to shrink the time between discovery and recovery and prevent attackers from using the same weapons in multiple attacks, Weed said.
"This is active defense," Bochman said. It goes far beyond cyber hygiene, such as the need for utility employees to steer clear of phishing email messages that could allow entry by a hacker. "Active defense assumes they’re in there, or they’re going to be," he said.
The strategy was patterned after the Aegis missile defense system on U.S. Navy warships, Bochman said. "If one missile is coming in, or two or three, you can take them out," he said. "But if it’s 150 missiles, humans don’t have the ability to track and take out those missiles."
"Aegis was developed to automate that process," Bochman explained, on a recent hourlong car ride from INL’s headquarters in Idaho Falls to the 890-square-mile INL desert test facility.
Budget reprieve
The Idaho laboratory programs were largely protected in the 2018 Energy and Water Development Appropriations bill, as House members in July rejected cuts in nuclear and cybersecurity programs called for by President Trump’s budget proposal in May. INL’s safeguards and security programs were allotted $133 million, an increase of $3.7 million over fiscal 2017.
The facility, approaching the size of Rhode Island, accommodated test firing of battleship guns in World War II and vital early nuclear reactor research (Energywire, Oct. 17).
Today, the INL site contains a working 61-mile high-voltage transmission loop, the Critical Infrastructure Test Range, with seven substations and a control room, for trials of a wide range of cyberattacks and grid resilience challenges.
Along one road, INL media representative Misty Benjamin pointed out a concrete slab that was the site of the 2007 "Aurora" test, when engineers succeeded in hacking into the controls of an electric generator and forced out-of-sync connections with the INL power grid. The generator blew up, like an auto transmission that is shoved into reverse at highway speed.
Elsewhere on the site is test bed for wireless communications technologies that play an increasing role in grid operations. The facility was built in a depression in the desert — the caldera from a volcanic eruption eons ago — which helps shield microwave transmission from outside interference.
A central part of the CES-21 strategy, Foster said, focuses on identified potential attack points in substations and control rooms. "Let’s see what is the most vulnerable area that may already have exploits written for it," she said. "If something were to happen, what would be the consequences to that utility? Do they care? Would it be a hard fix, an easy fix; long or short lead time? And then prioritize the issue.
"You’re getting a threat indicator that is machine readable, and you can create a remediation action to another machine," Foster continued. "So it could indicate, for example, that a back channel has been opened and you’d create a remediation action to either kill that session, or to kill the port the back channel was going on," or change the software in a system to eliminate the threat.
MMATR seeks to detect out-of-the-ordinary behavior in control systems, compare that against known threats, and create and test defenses against the threat. Understanding how and where attackers could do the most harm to grid systems would also guide development of defenses, advancing a utility’s chances of finding if an enemy has broken in. Machine learning — feedback loops that allow software threat assessments to grow more precise with experience — is part of the strategy.
"The attacker may be hiding in an embedded [grid] system and nobody is watching," Foster said. "That is the problem we’re trying to deal with. Right now, utilities can’t even answer that question because it’s a little, teeny, embedded thing that they incorporated when they created the system, and they never saw it."
A project milestone last year was creating a threat indicator language for the utilities in a version of the STIX programming language that would enable utilities, vendors and cyber analysts to label attack malware in common terms that can be shared at computer speed. The new version is particular to grid control systems.
More testing on the entire project is needed to prove the concepts on actual equipment, Foster said, "so then we’ll have a little more confidence that yes, this could work." Final activities could be wrapped up by the end of 2019, she said.
INL’s cybersecurity agenda points in many directions. To take one case, its "Sophia" defensive tool offers "fingerprinting" software to utility control room operators and other managers of industrial control systems, enabling them to map connections and network traffic in and out of the systems.
An industrial control system’s communications patterns, which normally are largely fixed and static, are mapped to define normal conditions. Then Sophia keeps running passively in the background to track communications across a utility network, looking for trouble.
Foster said her lab’s work on CES-21 is only a part of that large, diverse project (it also includes research on integrating renewable energy into California’s grid).
‘Consequence-driven’ defense
INL’s role of testing MMATR against actual components fits its identity as the prime DOE lab for applied engineering, said Wayne Austad, director of INL’s Cybercore Integration Center.
"If you just let the market drive it, [cybersecurity] innovations will be centered around a natural progression of what we use in IT [information technology]," rather than the operational technologies that run equipment in the grid’s physical domain, he said.
Austad said INL is pushing a defense strategy it calls "consequence-driven cyber-informed engineering." It seeks to calculate the most serious threats to critical infrastructure from particular advanced cyber adversaries and find out what would happen if the attacks succeeded. Specific defenses must be developed and shared, designed to prevent catastrophic damage.
"This process starts with identifying the highest impact, most severe consequences, and then discovers the best process design and protection approaches for engineering out the cyber risk," he said. In some cases, hardware barriers — not software "firewalls" — will be needed.
Getting inside attackers’ heads is crucial, he said. "You have to understand the ‘kill chain’ that the adversary would follow to target and take out the most critical systems," and head them off, Austad said.
"We should not be responding to an alert that some nation-state attacker has gathered this design information [about a target], and go, ‘Whoa, this important,’" he said. "We should already know that."
A top-level adversary can get in somewhere, Austad said. "You just have to live with that," he said. The challenge, as he sees it: "Keep them away from places where they can really hurt you."
