How Cloud and Fog computing will advance SCADA systems

Nov. 21, 2017 – In building management systems (BMS) and human machine interface (HMI), manufacturers need to seek the right architecture for supervisory systems. Costs are being further driven down from legacy systems while historically, reliability is improving and efficiency is increasing. Due to the enhanced connectivity of these systems today, improving cybersecurity has become an equally important goal.

For SCADA, one of the most important technology developments has been the trend away from serial communications to IP-based industrial network communications. This has allowed much more flexibility in where the supervisory server is physically located relative to the PLC. Remote Terminal Units (RTUs) were geographically distributed away from the SCADA host. This was very valuable in realizing distributed control and remote monitoring, but the limited bandwidth radio communications of the time made this architecture unsuitable for many SCADA applications.

Combining the advances in client-server technology and network technology, it has – for several years now – been possible to migrate the SCADA or BMS host server out of the control room and off the shop floor and into a physically secure, environmentally controlled data centre. In the lexicon of Cloud computing, when we have the supervisory host on a dedicated machine running in a data centre we have created a Private Cloud.

Consider this example. A grocery store chain has a warehouse where they store cheese products. They purchased an air quality control system to ensure the “cave” was kept at optimal conditions to maintain the quality of their cheese products. This environmental system consisted of PLCs supervised by HMI software. It was installed on a PC in the warehouse. The management team for warehouse operations was concerned that physical damage could occur to the computer, for example, a forklift accidentally running into the PC, which would cause them to lose control.

Since the PLCs used IP connections and the HMI was easily converted to a SCADA host with the HMI client, it was straightforward to move the SCADA/HMI to the IT data centre. This data centre is located at the headquarters of the company and was installed on a dedicated server. In the process of migration, the single-user HMI station was simply converted to a multiple-user SCADA system so that the maintenance team had access to the same information that the operations team was seeing.

With virtualization technology, it is not necessary to have the dedicated machine in the data centre. This is now becoming increasingly common as supervisory vendors make the necessary adjustments to allow both the client and server to be deployed on virtual machines (VMs). The next step in the warehouse migration project was to move off the dedicated server and onto a VM. The rational for the second move was that it is possible to restore a VM from a backed up version in approximately 15 minutes. In the data centre, IT had a common process to continuously backup the VMs on a frequent basis. For this customer, this was preferable to have a redundant backup because it was a familiar process for IT to manage.

This illustrates the growing role of IT in operational technology (OT) deployment. While some aspects of managing IT versus OT have different primary objectives, there are other aspects where IT best practices will benefit OT managers working in collaboration with IT, which includes developing a robust cybersecurity culture.

For IT, business risks are mainly related to the confidentiality and integrity of data processed and hosted by IT systems, which leads to intangible consequences, such as loss of know-how and loss of reputation for the enterprise. For OT systems, business risks are related to the availability, integrity, reliability and safety of the industrial control system itself. Risks include operational consequences in the physical world, such as production shutdown and financial losses, environmental damage and the inability to control the process or to obtain accurate information about its state.

Further collaboration and development of appropriate common practices and shared principles are the hallmark of top performing companies and has been enabled by the advancement of OT architectural thinking.

What about the public Cloud? While the technology is the same, the grocery store did not want to risk losing the system if access through the Internet service provider was lost. It was also concerned about an architecture requiring the use of the Internet to connect the PLCs to the SCADA host. The concerns were both from the perspective of latency and reliability. In some cases, data privacy is also a concern with having third-party hosting of the operational data. In this case, that was not a concern.

In a second example, the public Cloud is the ideal place to host the SCADA server. PcVue has worked with a service provider who deployed SCADA software on Amazon Web Services. In this case, the architecture consists of a private Windows Server running on a VM connected via a Distributed Network Protocol (DNP3) to devices that are strategically and, in some cases, very remotely located to provide real-time views of the voltage across a network of high voltage transmission lines. The DNP3 protocol is used to connect to monitor and control electrical devices and is very heavily used in North American substation automation.

In this case, the connection was made by creating a VPN running over satellite links to some very remote locations, far from the substation. The ability of the utility to monitor the transmission line voltage in real time offers great value. The satellite connection may have latencies that aren’t appropriate for substation automation, but when compared to putting a data logger on the transmission line in the field and collecting the data a week later, it is fast enough.

The utility could access the data with the HTML browser and a connection to a public server providing Remote Desktop Services (RDS) for each user. The RDS server runs on a VM connected to the data acquisition server through another VPN.

What is significant about these examples is that public and private Cloud architectures can both be the right choice for SCADA but one size does not fit all. In 2017, a lot of discussions focus on the Industrial Internet of Things (IIoT) and Industry 4.0. While Industry 4.0 goes beyond the scope of operations, within the OT space both concepts are driving toward Cloud-based solutions. In the past few years, the focus of discussion on IIoT is increasingly on data acquisition and Big Data analytics rather than supervisory control.

In 2014, it was difficult to find a real consensus of what IIoT is and what the architecture for it would be. Over the past three years, a common architecture has emerged — one in which field sensors are connected to gateways that move the data to a public Cloud where it is analyzed and individuals or software packages may access it with standard APIs, but not yet standard protocols. New low bandwidth networks are emerging from companies and alliances, and LTE and 5G cellular providers. What they have in common is the expectation that data rates will be slow—updating once a day or once an hour. Most are priced on the amount of data transmitted to the Cloud.

For many SCADA and BMS applications, this allows for sensor integration that was not previously possible in the past. For example, a private LoRaWan network can be constructed to integrate IIoT sensors without going to the Cloud but rather by directing the data from the gateway directly into the supervisory system.

In the parlance of the IIoT, this is known as edge computing, as in the edge of the Cloud. The term Fog computing was coined to recognize that some data treatments and response needed to be done near the process rather than from the Cloud. In the previous examples, the control of the warehouse was essentially Fog computing even though it didn’t make use of IIoT sensor data acquisition. The second example is clearly Cloud computing.

Wikipedia defines Fog computing as “providing location services and service quality for real-time applications and streaming. It also permits a bigger heterogeneity since it is connected to end-user devices and routers. The applications can include industrial automation, transportation and networks of sensors.” To most control engineers and SCADA vendors, this sounds very familiar.

As we develop the language of IIoT and incorporate it into the supervision of industrial processes, infrastructure, buildings, utilities and other common SCADA verticals, it is clear that there is no one solution that fits all. Software vendors who supply platforms for SCADA, HMI and BMS will need to continue to be flexible and scalable in terms of supporting various architectures for deployment, including Fog and Cloud computing, and incorporating IIoT data accessible from the Cloud and private IIoT networks which collect IIoT sensor data directly from a gateway device at the edge.

The location services aspect of Fog computing is essential to delivering the exponentially increasing data in a meaningful and actionable way to users of the supervisory system. A secure contextual mobility server – which may be Fog or Cloud hosted – that utilizes location services and user profiles to deliver intelligent data to the mobile devices of operators and maintenance teams, based on the user’s job responsibilities, is essential in driving down costs and increasing the efficiency and reliability of supervisory control for smart buildings, smart grid and emerging Industry 4.0 evolutions.

Ed Nugent has 24 years of experience with SCADA development and implementation and is currently the chief operating officer for PcVue Inc., www.pcvuesolutions.com, a global independent SCADA/HMI provider. His career has spanned education, engineering and management leveraging a passion for capturing and communicating the business value of measurement and control technology.

This article was originally published in the November/December 2017 issue of Manufacturing AUTOMATION.