Real-time computing is particularly important in manufacturing. Many industrial PCs (IPCs) rely on a real-time operating system (RTOS) or real-time extensions to operating systems to guarantee predictable, deterministic behavior for controlling machinery and processes.
Real-time systems must respond to inputs and events within very tight time constraints, typically within milliseconds or microseconds, to ensure safe and efficient operations. A deviation from this expected behavior, even for a fraction of a second, can have severe consequences.
Get your subscription to Control Design’s daily newsletter.
However, because the hardware and upper-level software is similar to standard office computers, and because these IPCs will likely be connected to a site’s computer network, making them both targets of and conduits for hacking, it is not uncommon for the information-technology (IT) department to add any computer, whether industrial or not, under its umbrella and, in doing so, do a disservice to the real-time capabilities of these systems by treating them the same as any other computer in the office.
Where IT departments impact industrial PCs
I’m not trying to condemn IT departments, but I have seen this same story many times—the design engineer spends many hours researching available real-time systems and IPCs that can provide marked improvements to the manufacturing process, only to have the IT department set the IPC up for automatic updates from the cloud. This can erase any gains to the manufacturing process, result in additional rework or downtime or even cause safety issues in the plant.
A significant risk posed by automatic software updates to industrial PCs is the potential for incompatibility between the updated software and the real-time extensions or operating systems. Real-time extensions provide the low-latency performance necessary for time-critical tasks in industrial settings. An update pushed without thorough compatibility testing may result in drivers, application programming interfaces (APIs) or other critical software components becoming incompatible with the real-time requirements of the IPC.
For example, an update to the operating-system kernel or a hardware driver may inadvertently affect how the real-time extensions interact with the hardware, causing delays or jitter in the execution of time-sensitive operations. This disruption could lead to processes being executed out of order, data being processed too late or even machinery being controlled incorrectly. Given that many industrial systems rely on precise timing and synchronization, even small disruptions can lead to system instability or failure.
Incompatible firmware and drivers
Another key issue when updating industrial PCs automatically is the potential for new drivers or firmware to be incompatible with legacy hardware. In manufacturing environments, many IPCs are deployed with specialized, sometimes custom, hardware designed to work with specific machines or sensors. These components often require specialized drivers that may not be updated as frequently as general-purpose drivers.
Automatic updates are often deployed with the assumption that they will improve system functionality and security. However, they can just as easily introduce instability or new bugs into the environment. IPCs are often running specialized software tailored to the unique requirements of a given manufacturing process. These systems are frequently optimized for performance, and even minor changes to the software can disrupt the delicate balance that allows them to function reliably.
Regulations and standards
Additionally, many industrial sectors are governed by strict regulatory standards that mandate specific operational conditions, software versions or configurations. Automatically pushing software updates can introduce significant compliance risks if the new software versions inadvertently cause systems to fail to meet these regulations. For example, if a software update causes a manufacturing process to operate outside the prescribed tolerances, the company could face non-compliance issues, resulting in fines, sanctions or legal action.
Best practices for industrial PC update management
Implementing a controlled update schedule: Instead of automatic updates, create a controlled update schedule where software updates are tested in a dedicated environment before being applied to production systems. This allows IT teams to ensure that updates do not disrupt real-time performance or introduce instability.
Thorough testing of real-time components: Any software update should be rigorously tested to ensure it does not interfere with real-time extensions, drivers or other time-sensitive components. Many times, the vendor of the real-time system is already doing this in-house and offering tested updates to the systems. Check with them first.
Backing up of systems: Before applying any software updates, ensure that full backups are created to allow for quick recovery in the event of a failure.
Establishing clear update protocols: Define clear protocols for handling updates, including user acceptance testing (UAT) and rollback plans in case a problem arises after an update.
How to handle software updates for industrial PCs
While automatic software updates may be a common practice in general IT environments, they pose significant risks to the stability and performance of industrial PCs in manufacturing settings. The potential for incompatible drivers, real-time extension disruptions, security vulnerabilities and unintended system instability means that updates must be carefully controlled and tested. For the sake of maintaining smooth, reliable and compliant operations, IT departments should carefully consider their update strategies and avoid automatic software pushes to industrial systems. Instead, updates should be planned, tested and validated in controlled environments to ensure that the unique requirements of industrial automation are met and maintained.
