How to assess industrial control system vulnerabilities

A Control Design reader writes: New industrial control system vulnerabilities are constantly popping up. It sometimes seems overwhelming. We’d like to be more proactive, but we’re integrating more robots/cobots into our workcells these days and keeping up with interoperability and controller interfaces is daunting enough. Is anyone else struggling to stay current? Are there guidelines or standards for cobot/machine integration and cybersecurity?

 

 

 

 

The feeling of being overwhelmed is understandable and common across manufacturing facilities in the Americas. These teams are dealing with more frequent and severe cyberattacks, largely due to the increasing number of connected devices like software and control units on the factory floor. Alongside this, there's pressure to boost production and cut costs, which can quickly become too much to handle. In such an environment, where operational technology and automation are essential for reducing costs, increasing production and minimizing cyber risks, it's crucial to have a clear strategy. Although standards like International Electrotechnical Commission (IEC) 62443 provide valuable industry insights, we will focus on broader automation guidelines that make compliance easier and improve interoperability. By proactively addressing security, automation, collaborative robots (cobots) and operational technology in line with business objectives, companies can reduce costs, enhance efficiency and improve operational resilience, all of which lead to greater customer satisfaction.

 

While every connected automation setup is different, three key factors increase the risk in its overall structure. These include the ability to identify vulnerabilities before and after deploying software, the broad impact of a security breach, and the speed at which a fix can be implemented before an attacker exploits the weakness. Although much has been discussed about these aspects, we suggest simplifying the architecture to manage these risks better, particularly through the use of all-in-one automation platforms. All-in-one platforms, while they do not completely eliminate the risks found in more complex systems, do simplify management of these components. This simplification helps production teams develop and maintain better cybersecurity practices with greater confidence. To further strengthen these practices, we recommend working with industry experts. This partnership can help ensure that the solutions consider these factors during product design, commissioning and operation, leading to clearer production expectations and enhanced quality assurance.

 

All-in-one automation platforms are designed with a single integrated development environment at their core, where security is built in. This design meets the stringent resiliency and quality management requirements of even the most demanding environments. Managing a single software package that is compatible with various device and controller firmware versions is simpler than handling multiple systems from different suppliers. This consolidation facilitates quicker updates and reduces compatibility issues. The software also helps operators by showing device firmware versions, available updates, release notes and other critical information to keep production lines operating efficiently.

 

All-in-one automation platforms streamline the management of devices, reducing the burden and enhancing both reactive and proactive cyber hygiene. These platforms use globally recognized industrial protocols to ensure interoperability and integrate the security features of common industry standards, benefiting both customers and suppliers. Industry-standard protocols undergo extensive external scrutiny, making them easier to update and secure at scale compared to proprietary systems.

 

All-in-one platforms enhance security management by documenting and backing up program versions and allowing for version locks. Updates can be applied swiftly during scheduled maintenance across all machines or production lines, accompanied by necessary change management documentation. With features like automatic software updates, these platforms help teams maintain control over factory floor programs with minimal disruption. Embracing a security-first approach may require a cultural shift, but partnering with industry experts who support and update their automation software can instill confidence in production teams.

 

Well-implemented all-in-one platforms simplify the integration of cobots into factory settings. These platforms support a wide range of firmware, reducing software management challenges and enhancing flexibility. They also ensure quick deployment of security updates, helping to close vulnerabilities more rapidly.

 

It's important to remember that you're not alone in this process. Manufacturers of all-in-one automation platforms adhere closely to industry standards to maintain interoperability and a unified software approach. These manufacturers are skilled at ensuring systems work seamlessly together without compromising security. While these are only guidelines, we recommend partnering with a provider that embodies these principles in their automation solutions. This partnership will prepare your team to identify vulnerabilities, limit their impact and deploy fixes confidently and swiftly.

 

 

 

 

Many are struggling in this rapidly changing landscape of cybersecurity in the industrial automation control systems (IACS) community and operational technology (OT) environments. There are multiple factors contributing to a complex and difficult problem. Added to the inherent challenges created by all the growth and innovation changes that are happening in all kinds of industries, which creates a backlog of work for interoperability and cybersecurity checking and adapting, there is also the issue that the industry is going through a transition from isolated networking environments to more feature-rich connected environments.

 

But who is going to be doing all this extra work? Companies are tightening their belts and working to cut overhead. Asking for additional and very specialized workers who are seen as purely overhead is a tough sell and rarely popular. However, it is important to reframe the work that needs to be done into its true role. Cybersecurity isn’t overhead; it is risk mitigation and avoidance. These individuals prevent legal liability by taking the required due diligence for the environment and equipment to be protected. They prevent entire plants and businesses from being shut down due to a cyberattack, or worse, having to pay a large ransom to criminals to get production back up and running. Even with cyber insurance, the impact of one of these attacks can be devastating to entire industries.

 

While paying ransom through insurance may prevent profit loss by getting back to work, it also incentivizes those attacks and others to just do it again, but now with additional funding to create more sophisticated attacks. Well-funded hackers and malicious actors require even more security and defensive measures to keep out, costing much more for the entire industry in the long run. Well-trained and qualified cybersecurity personnel are not just industry best practice, they can make the difference between a company being an industry leader or losing millions of dollars in a matter of days.

 

Also, as IACS environments embrace many of the innovations and features from traditional information technology (IT) environments, this opens up all the traditional attack paths that IT environments have been trying to defend for decades, but frequently without any of the defense structures. This change has caused an increase in malicious actors targeting these environments more than they have historically, because there are so many known vulnerabilities they can exploit. Trying to create a new “zero day” attack is much harder than doing an internet search for known vulnerabilities in a specific item, so any information an attacker can find out about your system can quickly lead to disaster, with almost no time to react. Without cybersecurity personnel working with designers and the day-to-day operations teams, implementing changes and protections against those known vulnerabilities, the risk of attack becomes more likely and more damaging very quickly. It is going to take a conscious investment to meet the challenges of today.

 

Currently, IEC 62443 is the only set of major cybersecurity standards for IACS type environments. This is true for a variety of industries, including robot- and cobot-filled manufacturing spaces. Many industries that do have regulations frequently have based their regulations off this set of standards, as well. Seeking to implement certification for your environment, or at least buying components that have some level of certification, can be a starting point for businesses that want to be better but just don’t know where to start.

 

I would also recommend asking suppliers or looking for suppliers who provide regular vulnerability notifications with mitigations and patches. These are proactive measures that can give focus to your efforts but are not a replacement for dedicated cybersecurity personnel who are trained and supported in looking to improve your security posture. Combining this with an emulated or fully digital testing environment to test patches and changes without taking down the actual production environment will help with the work of maintaining security without impacting day-to-day operations.

 

In a lot of ways cybersecurity is becoming more important than physical security because a connected environment is vulnerable to criminals worldwide, not just in your local area. And just like you wouldn’t leave the front door to your locations unlocked with no security guard, why are we not investing in security against an even larger group of attackers? Because we can see unlocked doors or holes in fences. We cannot see cybersecurity threats until you hire someone who knows where to look.

 

 

 

These comments hit on a big friction point in the cybersecurity community at the moment. Vulnerability fatigue is a major issue. At the moment, a lot of cybersecurity monitoring products really focus on asset management and vulnerability detection. The idea falls into the risk equation:

 

Risk = likelihood x Impact

 

But, if you break likelihood down a bit further, the likelihood of an incident comes down to threat x vulnerabilities:

 

Risk = business impact x vulnerabilities x threats

 

So, the idea is that, if you can reduce your vulnerabilities, you can reduce the risk of a cyber-related event. But this doesn't take into account the amount of resource allocation necessary to actually address all the vulnerabilities. A lot of the OT security vendors provide service offerings to help address this resource problem, but for most end users, the budgets for this topic are limited.

 

So, there's a new train of thought that chasing all the vulnerabilities is an exercise in futility. So, the idea is to look at the risk equation again. There are a couple of things to consider with the equation, as well. You can also reduce your risk by reducing the business impact of an event. So, if you build into your system fail-safes, that can mean the system is operated even when certain parts are affected by a cyberattack, then you've reduced the risk, as well. Another point is that vulnerabilities can be addressed in multiple ways. Patching is always the one that comes to mind, but using firewalls and endpoint protection can also "remove" vulnerabilities from affected systems by removing the attack vector.

 

Finally, the biggest point is that every organization needs to determine what its acceptable risk level is. You can't always address all the risks because of limited resources. Every organization has to draw the line at: "This is acceptable for us." That could be that they only patch biannually or that they don't patch regularly, but instead put firewalls in place. That risk acceptance can change as the available resources change. Cybersecurity is a constantly changing and evaluated program.

 

If they are looking for reference stands, I'd start with the latest version of the National Institute of Standards and Technology (NIST) Cybersecurity Framework (CSF). It was released last year, and there's new guidance on industrial control systems (ICS). The NIST CSF is a bit more approachable than IEC 62443, but if they want a more comprehensive program, then IEC 62443 is the way to go. In both of these standards, there is a section that outlines picking your target "security level." This is essentially picking your benchmark for risk acceptance. What are you willing to do and not do based on the risks?

 

 

 

Typical guidelines in the past had been to isolate machines into separate networks to limit the potential vulnerability plantwide by using virtual private networks (VPNs). VPNs help with encrypting data transmitted, as well as with authentication to make sure only authorized users can connect with the machine. VPNs help secure machines from outside access of the network. The same integration can be done locally on the machine, as well. This involves making sure each network module on the machine has the same encryption of the data, as well as authentication access to the module. Setting up correct authorization per user, as well as continuously monitoring the activity of each module, can help with local security as well. Finally, if there is a security breach, making sure there is a plan for backup and recovery of the data is important. This will help ensure everything gets back up and running quickly as smoothly as possible.

 

 

 

This is a common issue with growing companies struggling to meet production goals while maintaining cybersecurity policies. The simple, but not easy, solution is utilizing original equipment manufacturer (OEM) or integrator services as an auxiliary to your current work force. Simple, as in the idea of contracting these services will free up your workforce to meet production goals. However, many of us know how difficult it is sometimes getting additional funding approved.

 

In the same manner you’re using cobots to augment your production force, OEM tech support can provide myriad maintenance services on the equipment they know best so your team can focus on output and quality control. The maintenance services offered by many OEM providers also include cybersecurity services, such as operating system (OS) and firmware patching, system hardening and secure integration between platforms.

 

One guideline for patch management within an OT network can be found in the International Society of Automation (ISA)/IEC 62443-2-3 standard. It outlines ways to logically design an OT patch management program for updates and logging. It’s also a good idea to include input from your system OEM provider because they test patches prior to release and can also provide guidance on which patches should be installed.

 

 

 

Vulnerability management is a large and important topic in both the IT and OT environments. IT has been addressing this issue for multiple decades, while it is a much newer issue in the OT space due to the increase in attacks that now directly target OT systems. In fact, in the past, the automation world actively discouraged patching or updating unless absolutely necessary; this now must change to address the incredible growth of cybersecurity threats to OT assets.

 

From our perspective, we do see many customers struggling to stay current. However, there are also several standards, guidelines and frameworks that can help. The key takeaways are that this is a manageable effort that will take time and resources to create or change processes and train personnel to accomplish. Perhaps the most important takeaway is that OT cybersecurity in general and vulnerability management specifically must be planned as a continuous effort with no anticipated end—much like we have come to view the quest for a safe plant environment.

 

On the general cobot/machine integration topic there are multiple industry-wide efforts to ease the integration of all types of machines or sub-systems into a manufacturing control system. One effort gaining traction with many control system OEMs is the module type package (MTP), based on the VDI/VDE/NAMUR 2658 standard, which aims to simplify integrating any sub-system to a distributed control system (DCS). Then specifically for cobots, there is a NIST publication—NIST Advanced Manufacturing Series 100-41—Best Practices for the Integration of Collaborative Robots into Workcells Within Small and Medium-Sized Manufacturing Operations. Additionally, most, if not all, of the cobot OEMs offer guides on how to best integrate cobots into manufacturing operations.

 

Now back to the vulnerability management issue. NIST and ISA/IEC created separate OT cybersecurity standards, which have effectively coalesced into a single body of knowledge over the last 20+ years. The ISA99 standards committee initially created the basis of what became the ISA/IEC 62443 international series of standards, which address what cybersecurity technologies can be employed, how to employ them, how to manage installed systems, how service providers should prepare themselves and how OEMs should securely manage their product lifecycles and certify individual products.

 

The NIST 800-82r3—Guide to Operational Technology (OT) Security is used more for government related projects in the U.S. like defense industrial base and regulated utilities. However, the NIST Cybersecurity Framework (CSF) V2.0 is used extensively for evaluating current state and planning next steps in a cybersecurity program with a risk-based approach.

 

All the cybersecurity standards consider cybersecurity “hygiene” to be one of the core aspects of cybersecurity. This means following basic guidelines like individual logins with complex passwords that are regularly rotated and keeping the firmware and software current. So how do we do this?

 

There are three tiers of vulnerability management that could be described as manual, semi-automated and automated. Manual processes include creating an OT asset inventory list and then subscribing to multiple vulnerability notification systems. Those systems could be government-based, like the National Vulnerability Database (NVD), or vendor-provided, which may give better and more detailed information but will require one subscription per vendor.

The semi-automated systems will take an inventory list and find all the vulnerabilities and may allow for some management of the remediation process. Finally, a fully automatic system strives to identify the vulnerabilities, download the fix (new firmware or software service pack), present the queued remediations for approval and then deploy the fix automatically. The fully automated systems are typically deployed in an IT environment and have not found much adoption in OT due to a need for quality control with most OT organizations requiring in-depth testing, production coordinated scheduling and management of change processes.  

 

The previously mentioned standards also advocate for many other types of cybersecurity controls to implement a defense-in-depth style of protection with multiple overlapping layers. These other layers of protection might include deep packet inspection (DPI) firewalls, segmentation of different production areas, separation of IT and OT networks or intrusion detection systems (IDS). Most importantly, it is critical for long-term success to approach cybersecurity with a continuous-improvement, programmatic methodology, much like industrial plants approach human safety. So, while vulnerability management is a key part of an OT cybersecurity program, don’t forget to consider and implement other OT cybersecurity protections.