It was a near miss for critical infrastructure that briefly captured the news cycle.
On February 5, 2021, an employee at the water treatment plant in Oldsmar, Florida, noticed a cursor moving on one of the facility’s computer displays. The employee initially thought a supervisor had remoted into the plant’s network but grew concerned when he realized someone was attempting to release toxic quantities of a common treatment chemical into the water supply. The employee took successful action to protect a system that serves the city’s 15,000 residents.
In the aftermath, the city manager got involved, as did the mayor, sheriff, and Federal Bureau of Investigation. Officials called a press conference to discuss the incident. Though some reports sensationalized the threat level – routine testing measures would have prevented anyone from being poisoned – the incident highlighted the potential vulnerability of critical infrastructure to cyber intrusions.
Experts have long warned of cyber threats to the water industry, even as breaches or ransomware attacks to the financial services sector have become routine. Just as we need to strengthen cybersecurity for information technology (IT), we must also implement better protections for the operational technology (OT) that runs our critical infrastructure.
Though they use similar equipment and networks, IT and OT systems have different security and operational needs. For a long time, the rule of thumb for OT was “if it ain’t broke, don’t fix it.” In today’s climate of connectivity, automation, and remote operation, that rule no longer applies.
Then again, the IT security playbook of patching, upgrading, installing antivirus software, and password expiration is like the cyber hygiene approach of brushing your teeth or taking a shower. All are excellent practices, but they are generic and more difficult to accomplish in the OT environment because of cost, process disruption, and complexity.
For OT systems, a risk-based approach offers a more cost-effective way to protect their underlying critical assets, including supervisory control and data acquisition (SCADA) systems that operate utilities. It is impossible to eliminate risk, but it is possible to reduce risk to an acceptable level.
When evaluating the cyber readiness of an industrial control system, it’s helpful to express risk as an equation of likelihood multiplied by consequence. Reduce either the likelihood of a negative outcome, or its consequence – the total damage or disruption that could occur – and you reduce overall risk. Applying this equation to a SCADA system allows utility operators to deploy limited resources as efficiently as possible.
Undergoing a risk analysis is one of several practical steps utility operators can take to identify and shore up areas of greatest vulnerability. Other steps include the following:
- Inventory assets. You can’t manage what you don’t know you have. Make sure the inventory includes both hardware and software. Document network configuration, software versions, services packs, firmware, etc.
- Tighten remote access. The pandemic made working remotely a necessity, but some remote systems are more secure than others. Always use multifactor identification for any remote access application.
- Require passwords. Require them to be complex and activate them for all devices that control protocol gateways, instruments, switches, and routers. Never use default passwords.
- Control internet access. SCADA computers should not be used for internet browsing and email. Dedicate a separate computer for these tasks, preferably one that is isolated from the SCADA network. One of the most common ways for ransomware to enter a system is via email.
Defense in depth is another piece of armor that can help defend an OT system. It involves segmentation of the networks so that a bad actor must defeat multiple levels of security before gaining control of a system. The most well-known example of such a control hierarchy is the Purdue Model, which seeks to contain a system compromise to a single level of the network – ideally far removed from the most critical parts of the control system. Industrial control systems with large, flat networks could allow full access to intruders who manage to kick in the front door.
A related component of the risk-based approach is resiliency – a measure of how well your system can withstand a cyber event. Increased resiliency represents a broad, holistic approach to further reduce risk. Resiliency consists of non-computer defense elements such as staff training, local controls, hardwired interlocks, backups, and notifications separate from SCADA. In the Oldsmar example, the employee who acted on the moving cursor and the plant’s routine water testing provided crucial resiliency to the system.
To date the water industry has managed to escape any seriously damaging events, in part because of improved cyber practices but most likely due to lack of interest by would be attackers. Thus far, the worst part has probably been the negative attention to the Oldsmar incident by the media and others who may not fully understand the reality of the situation.
Though there have been discussions, no cyber regulations governing water utilities exist today. I believe they will eventually come, but these regulations will take months or years to have a significant effect. The cyber world moves much faster than this.
The threats are real, and so are the steps you can take now to reduce risk and better protect your community. The more sophisticated OT systems become, the more vulnerable they will be. It is up to all of us to act now and not wait for the next Oldsmar.