IP Packet Switching
Chapter 2, “IP Addressing,” explained that devices on the same subnet could communicate directly with each other without the need of a router. The second layer of the OSI model, the data link layer, handles addressing beneath the IP protocol stack so that communication is directed between hosts. Network packets include the Layer 2 addressing with unique source and destination addresses for that segment. Ethernet commonly uses MAC addresses, and other data link layer protocols such as Frame Relay use an entirely different method of Layer 2 addressing.
The first routers would receive a packet, remove the Layer 2 information, and verify that the route exists for the destination IP address. If a matching route could not be found, the packet was dropped. If a matching route was found, the router would identify it and add new Layer 2 information to the packet. The Layer 2 source address would be the router’s outbound interface, and the destination information would be next hop’s Layer 2 address.
Figure 3-7 illustrates the concept where PC A is sending a packet to PC B via Ethernet connection to R1. PC A sends the packet to R1’s MAC address of 00:C1:5C: 00:00:02. R1 receives the packet, removes the Layer 2 information, and looks for a route to the 192.168.2.2 address. R1 identifies that connectivity to the 192.168.2.2 IP address is through Gigabit Ethernet 0/1. R1 adds the Layer 2 source address using its Gigabit Ethernet 0/1’s MAC address 00:C1:5C:00:00:03 and a destination address for PC B of 00:00:00:00:00:04.
)
Figure 3-7 Layer 2 Addressing
Advancement in technologies has streamlined the process so that routers do not remove and add the Layer 2 addressing but simply rewrites them. IP packet switching or IP packet forwarding is the faster process of receiving an IP packet on an input interface and making a decision of whether to forward the packet to an output interface or drop it. This process is simple and streamlined for a router to be able to forward large amounts of packets.
When the first Cisco routers were developed, they used a mechanism called process switching to switch the packets through the routers. As network devices evolved, Cisco created Fast Switching and Cisco Express Forwarding (CEF) to optimize the switching process for the routers to be able to handle larger packet volumes. Fast Switching is deprecated in newer IOS releases and is not covered in this book.
Process Switching
Process switching, also referred to as software switching or slow path, is the switching mechanism in which the general-purpose CPU on a router is in charge of packet switching. In IOS, the ip_input process runs on the general-purpose CPU for processing incoming IP packets. Process switching is the fallback for CEF because it is dedicated for processing punted IP packets when they cannot be switched by CEF.
In IOS XR, the Network Input/Output (NetIO) process is the equivalent to the IOS ip_input process and is responsible for forwarding packets in software.
The type of packets that require software handling for both IOS and IOS XR include the following:
- Packets sourced or destined to the router (that is, control traffic, routing protocols)
- Packets that are too complex for the hardware to handle (that is, IP packets with IP options)
- Packets that require extra information that is not currently known (that is, Address Resolution Protocol [ARP] resolution, and so on)
Figure 3-8 illustrates how a packet that cannot be CEF switched is punted to the CPU for processing. The ip_input process consults the routing table and ARP table to obtain the next-hop router’s IP address, outgoing interface, and MAC address. It then overwrites the destination MAC address of the packet with the next-hop router’s MAC address, overwrites the source MAC address with the MAC address of the outgoing Layer 3 interface, decrements the IP Time-To-Live (TTL) field, recomputes the IP header checksum, and finally delivers the packet to the next-hop router.
)
Figure 3-8 Process Switching
The routing table, also known as the Routing Information Base (RIB), is built from information obtained from dynamic routing protocols, directly connected and static routes. The ARP table is built from information obtained from the ARP protocol. The ARP protocol is used by IP hosts to dynamically learn the MAC address of other IP hosts on the same subnet. For example, an IP host that needs to perform address resolution for another IP host connected by Ethernet can send an ARP request using a LAN broadcast address, and it then waits for an ARP reply from the IP host. The ARP reply includes the required Layer 2 physical MAC address information.
Cisco Express Forwarding
Cisco Express Forwarding (CEF) is a Cisco proprietary switching mechanism developed to keep up with the demands of evolving network infrastructures. It has been the default switching mechanism on most Cisco platforms that do all their packet switching using the general-purpose CPU (software based routers) since the 1990s, and it is the default switching mechanism used by all Cisco platforms that use specialized application specific integrated circuits (ASICs) and network processing units (NPUs) for high packet throughput (hardware-based routers).
The general-purpose CPU on the software-based and hardware-based routers is similar and perform all the same functions, the difference being that on software based routers the general-purpose CPU is in charge of all operations, including CEF switching (software CEF), and the hardware-based routers do CEF switching using forwarding engines that are implemented in specialized ASICs, TCAMs, and NPUs (hardware CEF). Forwarding engines provide the packet switching, forwarding, and route lookup capability to routers.
Given the low cost of the general-purpose CPUs, the price point of software-based routers will be much more affordable, but at the expense of total packet throughput.
When a route processor (RP) engine is equipped with a forwarding engine so that it can make all the packet switching decisions, this is known as a centralized forwarding architecture. If the line cards are equipped with forwarding engines so that they can make packet switching decision without intervention of the RP, this is known as a distributed forwarding architecture.
For a centralized forwarding architecture, when a packet is received on the ingress line card, it is transmitted to the forwarding engine on the RP. The forwarding engine examines the packet’s headers and determines that the packet will be sent out a port on the egress line card, and forwards the packet to the egress line card to be forwarded.
For a distributed forwarding architecture, when a packet is received on the ingress line card, it is transmitted to the local forwarding engine. The forwarding engine performs a packet lookup, and if it determines that the outbound interface is local, it forwards the packet out a local interface. If the outbound interface is located on a different line card, the packet is sent across the switch fabric, also known as the backplane, directly to the egress line card, bypassing the RP.
Figure 3-9 illustrates a packet flowing across a centralized and a distributed forwarding architecture.
)
Figure 3-9 Centralized Versus Distributed Forwarding Architectures
Software CEF
Software CEF, also known as the software Forwarding Information Base (FIB), consists of the following components:
- Forwarding Information Base: The FIB is built directly from the routing table and contains the next-hop IP address for each destination IP in the network. It keeps a mirror image of the forwarding information contained in the IP routing table. When a routing or topology change occurs in the network, the IP routing table is updated, and these changes are reflected in the FIB. CEF uses the FIB to make IP destination prefix-based switching decisions
- Adjacency table: The adjacency table is also known as the Adjacency Information Base (AIB). It contains the MAC addresses and egress interfaces of all directly connected next hops, and it is populated with data from the ARP table and other Layer 2 protocol tables (that is, Frame Relay map tables).
Figure 3-10 illustrates how the CEF table is built from the routing table and the ARP table and how a packet is CEF switched through the router. When an IP packet is received, if there is a valid FIB and adjacency table entry for it, the router overwrites the destination MAC address of the packet with the next hop router’s MAC address, overwrites the source MAC address with the MAC address of the outgoing Layer 3 interface, decrements IP TTL field, recomputes the IP header checksum, and finally delivers the packet to the next-hop router.
)
Figure 3-10 CEF Switching
Hardware CEF
The ASICs in hardware-based routers have a very high cost to design, produce, and troubleshoot. ASICs allow for very high packet rates, but the trade-off is that they are limited in their functionality because they are hardwired to perform specific tasks. There are routers equipped with NPUs that are designed to overcome the inflexibility of ASICs. Unlike ASICs, NPUs are programmable, and their firmware can be changed with relative ease.
The main advantage of the distributed forwarding architectures is that the packet throughput performance is greatly improved by offloading the packet switching responsibilities to the line cards. Packet switching in distributed architecture platforms is done via distributed CEF (dCEF), which is a mechanism in which the CEF data structures are downloaded to forwarding ASICs and the CPUs of all line cards so that they can participate in packet switching; this allows for the switching to be done at the distributed level, thus increasing the packet throughput of the router.
Software CEF in hardware-based platforms is not used to do packet switching as in software-based platforms; instead, it is used to program the hardware CEF, as shown in Figure 3-11.
)
Figure 3-11 dCEF Hardware Switching
Figure 3-11 also illustrates how the RIB process interacts with the RIBs of the routing protocols. The RIB process is in charge of the calculation of best paths, alternative paths, and the redistribution from different protocols and all these details merge into the global RIB (gRIB), where the best path for a destination network is installed. This is further distributed into the software CEF tables of different line cards, which is further mirrored into hardware CEF. The Switch Fabric is the backplane for all modules in the system. It creates a dedicated connection between all line cards and the route processors and provides fast data switching transmission between them.
In most distributed architecture platforms, if the incoming packet is control plane traffic or management traffic it is punted to the RP’s CPU. The following list includes some examples of packets that are typically punted for processing by the RP’s CPU or line card’s CPU:
- Control traffic, such as BGP, OSPF, IS-IS, PIM, IGMP, and so on
- Management traffic, such as Telnet, SSH, SNMP, and so on
- Layer 2 mechanisms, such as CDP, ARP, LACP PDU, BFD, and so on
- Fragmentation, DF bit set, IP options set
- TTL expired
- ICMP echo request