Multicast in Utility Networks
Before getting into the details of multicast utility design, let’s review the design blocks of a utility environment, as shown in Figure 5-18.
)
Figure 5-18 Utility Network Design Blocks
The WANs of most of utility environments are privately owned. This is one of the reasons for the legacy utility designs to have a WAN infrastructure for corporate function and a separate infrastructure for utility function (such as substations, grid control center, and so on). However, in many modern enterprise networks, it is common to see a Multiprotocol Label Switching (MPLS) Layer 3 VPN or L2 VPN deployed in the WAN block.
The reason for the change in the WAN design is that interconnected power delivery networks require close control of grid operations by increasing the visibility and creating smarter monitoring systems. Increased efficiency is created by using newer Supervisory Control and Data Acquisition (SCADA) measurements—typically every few seconds—to feed command, control, and monitoring systems, which causes an increase in bandwidth utilization. Several power system disturbances and extended outages have shown that the SCADA measurement data is no longer sufficient to provide adequate situational awareness to have visibility into the complexity of the interconnected power delivery networks. The goal in the next-generation power grid monitoring systems is to have visibility to small disturbances in these power delivery systems to improve the efficiency and fault tolerance of the power grid. Thereby, the need for newer measurement systems at different levels has created a requirement to modernize the network.
Field area networks are located at the distribution level and are broken into two tiers:
Distribution Level 2 tier: This is the last “last-mile,” also known as the neighborhood area network (NAN) level. Some of the functions of this network service include metering, distribution automation, and public infrastructure for electric vehicle charging.
Distribution Level 1 tier: This tier supports multiple services that aggregate and provide backhaul connectivity directly back to control centers, using system control directly with primary distribution substations. This tier also provides peer-to-peer connectivity for field area networks (FANs).
The deployment of FANs involves IPv6 and Wi-Fi mesh technologies as follows:
Substation tier: A substation level services wide-ranging scenarios—from connecting secondary stations, to complex requirements on primary substations—to provide critical low-latency functions such as teleprotection. Within the substation, networks may comprise from one to three buses (system, process, and multiservice).
Utility control center: This architecture block covers networks inside utility data centers and control centers. The control centers have very different requirements for security and connections to real-time systems. The use of firewalls at different tiers for regulatory compliance and protection of grid assets creates a requirement for multicast transport across the firewalls.
The data center network, or corporate office, follows the traditional enterprise environments and feature design components. To design multicast in a utility environment, you need to understand a couple key applications that are unique to utility environments, as discussed in the following sections.
PMU
Phasor measurement units (PMUs) allow for granular collection of important operational data to provide high-quality observation and control of the power system as it responds to supply and demand fluctuations. PMU data is useful for early detection of disturbances and needs to be collected at significantly higher frequency (typically 120 to 200 times per second); it requires a high degree of performance collection, aggregation, dissemination, and management. This requires higher bandwidth and data transmission with the least amount of latency. The bandwidth ranges between 1Mb/s and 2Mb/s of continuous streaming, and the latency component can be anywhere between 60 ms to 160 ms.
Most networks in the utility space are not designed or positioned to deal with the explosion of data that PMUs generate. PMU data is a multicast feed that is sourced from sensors and sent to the control grid. The design for PIM-SM or PIM-SSM depends on the number of multicast state entries and the type of IGMP driver supported in the host. The critical design factor to consider is the firewall’s support for multicast; most of the sensor data is within the substation domain, which is considered a North American Electric Reliability Corporation (NERC) asset. The control grid is also protected by a firewall infrastructure. For PMU application, multicast transport across the firewall infrastructure is a key design factor.
Radio over IP Design
In the utility environment, the Radio over IP design is a key building block required for communication. The use of multicast needs to be understood based on the design of Radio over IP systems. The communication details are as follows:
Individual calls between two radios from different site (Unicast could be a possible communication mechanism.)
Group calls from radios from different site (Multicast could be a possible communication mechanism.)
Dispatcher calls between different sites
Radio to telephone calls, where the radio and the private automatic branch exchange (PABX) are at different sites
Normally, Radio over IP design is considered any-to-any multicast. You are correct if you are thinking about bidirectional multicast design.
In addition, the energy grid application utility infrastructure hosts corporate applications such as music on hold (MoH), wireless multicast, multicast applications for imaging, data center requirements for multicast, and so on. The utility multicast design has requirements of different types of multicast traffic with separate domains. These domains can be virtual and separated by a control plane (MPLS Layer 3 VPNs using MVPNs or other methods) or can be logical, existing in the same control plane and overlapping. This is generally applicable to the utility WAN, control grid, and data center design blocks. It is important to configure multicast applications with a proper multicast addressing plan that can be rolled out enterprisewide.