Public Power and Cyber Security
This article first appeared in the July 2012 issue of the Northwest Public Power Association’s BULLETIN Magazine.
We have tremendously short memories. We tend to live in the moment and surround ourselves with news about what is happening in the here and now. Occasionally a catastrophic event may hold our attention, at least partially, for a year or two. But eventually the pain dulls, the trauma fades, and our focus shifts to something else. We likely made a few changes to business practices and regulations along the way, but it doesn’t take long for complacency to begin lulling us to sleep again. That is, of course, until the next catastrophe grips our attention and the process begins again.
In our utility operations we make life-and-death decisions on a daily basis. We learn from our mistakes, evaluate our incidents, and continuously work toward discovering better ways to stay safe. With over a hundred years of experience, we have learned that it takes constant training, vigilant awareness, formal accountability, and a never-ending search for safer technologies to keep our people out of harm’s way. As an industry, we know we can’t afford to do anything less than make safety our number one priority; we have embraced this and made it part of our culture.
We are really good at being safe and saving lives, because we learned from real events and we had to grow. We shouldn’t wait for a real event to occur in order to recognize that the same dedication and commitment to safety should be applied to security and — for the same reasons with a different flavor — protecting our way of life.
Although we can’t necessarily give cyber security that same level of concern, a similar approach still makes sense. There is a common misconception that security is something that can be put into place and then forgotten; that somehow security can simply happen automatically in the background without active participation. All that is needed is a few card readers, some firewalls installed, and voila! We are secure. The reality is that cyber threats, like safety, are not problems to be solved; rather, they are risks to be managed.
Management of Dynamic Risk
Our industry is quite adept at managing operational risks due to weather, natural disaster, equipment failure, and human error. The nature and extent of these events are largely predictable. Accordingly, effective response plans have been developed, and systems are appropriately engineered for reliability in the face of known hazards. Years of incident response, both practiced and real, have refined these response plans to some of the finest disaster recovery programs around.
Security (both cyber and physical) adds a variable to the equation which is largely absent from the common disaster recovery scenarios: intentional and intelligent adversaries.
Patrick Miller, president and CEO of EnergySec, has said, “Mother Nature may be cruel at times, but she is never malicious. Hurricanes, tornados, fires, floods, earthquakes, tsunamis, and even solar flares may affect us in new or unforeseen ways, but even these great forces of nature have their limits, and they do not adapt intentionally or intelligently to inflict greater harm, or circumvent our defenses. In most cases, we have the ability to predict when the storm, fire, or flood may subside; however, weathering a security incident is a different story. Depending on the resources and adaptability of the malicious adversary, the duration is largely unknown.”
We can debate the probability (or improbability) and motivations of potential attackers, but what is certain is that the continued deployment of modern, interconnected technologies in all areas of the power grid are greatly increasing our vulnerability and the potential for catastrophic attacks against our electric infrastructure. We do know that a cyber-event in today’s connected control system networks can possibly lead to a physical event. The Idaho National Lab demonstration named Aurora showed us that it is possible for a cyber-attack to physically damage a generator by manipulating in and out of phase settings at a rapid pace. More recently, the cyber-attack known as Stuxnet demonstrated that real-world physical damage to industrial control systems via malicious software is not just theoretical.
Understanding the Threat
Cyber security threats are often misunderstood, and frequently mis-characterized. The NERC CIP standards are an important starting point for industry with its focus on the most critical assets. Unfortunately, this focus is often taken as an indication that smaller organizations and ancillary systems are not important when considering security defenses. Likewise, the undue emphasis by media (and others) on cyber Pearl Harbor and similar catastrophic scenarios distracts from the numerous mundane, yet very real, possibilities that would carry significant impact to our industry.
Safety programs do not focus exclusively on life-threatening risks. Attention is given to all situations which could lead to personal injury or lost time. Even quality of life issues are addressed in areas such as repetitive motion and hearing loss. The focus is on ensuring the health and well being of our employees and customers.
Likewise, security programs must focus on the overall reliability of our operations in support of the customers we serve. This means we must move beyond simply protecting against doomsday scenarios, and focus on assuring that our technology systems reliably support our operations as intended. It should be seen as unacceptable for an attacker to disable or disrupt any aspect of our operations, whether it be an entire interconnection, a distribution feeder, or your neighbor’s smart meter.
Another challenge in developing a proper understanding of security threats is the dearth of real-world incidents. Truthfully, we have not yet seen cyber threats and vulnerabilities materialize into incidents of significant material impact. Yet this does not mean it won’t happen.
Along the U.S. West Coast, billions of dollars are being spent to retrofit buildings, prepare response plans, and conduct emergency drills in anticipation of a major earthquake in the Cascadia Subduction zone. We have not yet experienced such a quake, but we know we will. Likewise, most utilities have not yet felt the sting of a cyber-incident within their operational systems. Yet, we can see the tectonic shifts occurring in our infrastructure, and know that security incidents are inevitable.
Creating a Culture of Security
We have placed many remote solutions into our control environments over the last few decades to help manage the technologies we deploy in our plants, substations, and control rooms. Automation has increased efficiency and visibility, but it has also created thousands of new connections that didn’t exist before. The modernization of the grid is integrating new technology at an unprecedented pace which is making our operation and communication systems more complex every day. Whether intentional or accidental, the potential scope of impact due to the interconnected systems is growing exponentially.
With the increasing vulnerability of our power delivery systems technology, it is essential that we continue to address the security of these systems. The NERC CIP Standards, though not perfect, do provide a foundation upon which the industry has begun to build security programs; however, there are many that do not meet the threshold for applicability. In fact, roughly 97 percent of U.S. electric circuit miles are not covered by NERC CIP, leaving some of the bulk electric system, most of the distribution system, and nearly all of the smart grid elements out of scope. This is not to suggest that those elements should be brought in scope for NERC CIP. Rather it is to remind us that protecting our power delivery systems doesn’t come from following a set of regulatory requirements; it is born out of a collective desire by the industry to be secure.
Just like safety, good security requires support from all levels of an organization; and in the case of an interconnected system, the entire industry. Without genuine commitment, any rule, regulation, or policy becomes a mere compliance exercise. While compliance may be sufficient for simple, easy-to-measure actions, it is insufficient for complex, dynamic issues like safety or security. A strong culture of safety emphasizes that the rules are a starting point, but routine vigilance and attention are also required to identify and resolve issues or situations that may not have been known when the rules were developed. Likewise, security requires constant diligence to maintain a strong defense against the rapidly changing landscape of threats and vulnerabilities.
Although rules are important, they are not sufficient. We can prescribe action, but not attitude; and attitude is the secret sauce of successful security programs. Attitude must be developed, nurtured, recognized, and rewarded; it cannot be simply mandated. When it comes to safety, we can say we have nurtured and developed a positive attitude to do everything we can to be safe. However, that is currently not the attitude adopted by the electric sector when it comes to cyber security. Our industry at large hasn’t yet cultivated the positive attitude towards security. Perhaps that is because we can’t easily see the pain and suffering and we don’t have a hundred years of lessons learned from bad things happening. We have snippets of what ifs and some potentially close calls. But we don’t have a continuous stream of heinous acts that can result in collateral damage or — even worse — the loss of life. So it is more difficult to provide examples and lessons learned when it comes to cyber security because it is a newer discipline for utilities.
Partnering Security and Safety
In a recent informal poll during an event focusing on grid modernization for municipal utilities, only three respondents indicated they knew who was in charge of cyber security for their organization. This is in contrast to the unanimous show of hands for those who knew who was in charge of safety. Safety has been inculcated into our utility culture. The linemen can probably get service reinstated faster if they weren’t required to follow all of the safety protocols, but the risks of this approach outweigh the benefits. A similar mindset should be applied to security practices. The risks to our systems, customers, and employees are at stake. We cannot afford to take security risks any more than we can afford to take safety risks. Security supports safety. An organization that is both secure and safe will be more reliable.
Security can be integrated into the existing culture of an organization in a number of ways. One of the easiest is to use it as a compliment to your already successful safety program. Routine safety meetings can become security and safety meetings. Unions can adopt a motto of being Safe and Secure. Security awareness campaigns should be as robust as safety awareness initiatives and, whenever possible, be integrated. General cyber security training can be made available to all staff, not just those who have logical or unescorted physical access to critical cyber assets as NERC CIP requires. Meetings can begin with a security tip along with a safety tip. Many of our common safety slogans can also apply to security which may help to integrate cyber security into an organization more quickly.
Cyber security seems to be daunting at times, but there are numerous resources out there that can help. We can apply the lessons learned from the financial sector and others who have more experience dealing with cyber incidents. We can share information with each other using venues like the EnergySec-run National Electric Sector Cyber Security Organization (NESCO), which provides information and resources regarding cyber security, best practices, threats, and vulnerabilities. The NESCO Tactical Analysis Center synthesizes threat and vulnerability data from a variety of sources and provides relevant and timely information back to utilities.
The North American power grid has long been dubbed the greatest engineering feat of the 20th century. It has also been called the most complex machine on Earth. As we continue the journey of grid modernization, this complexity will only increase. The smart(er) grid will build on the pioneering work of our forebears. We must honor that legacy by ensuring that the technology we implement to improve the grid does not become its Achilles heel. Making cyber security a core element of our culture is a critical step towards meeting the challenges of operations in the cyber age.
Lisa M. Carrington is the Director, Program Office at EnergySec, a private, non-profit forum of information security, physical security, audit, and disaster recovery and business continuity professionals from energy industry utilities. EnergySec is also home to the National Electric Sector Cybersecurity Organization. Carrington can be contacted at either firstname.lastname@example.org or (503) 446-2000.