VLAN Routing with Layer 3 Switch Routed Ports
When Layer 3 switches use SVIs, the physical interfaces on the switches act like they always have: as Layer 2 interfaces. That is, the physical interfaces receive Ethernet frames. The switch learns the source MAC address of the frame, and the switch forwards the frame based on the destination MAC address. To perform routing, any Ethernet frames destined for any of the SVI interface MAC addresses trigger the processing of the Layer 2 switching logic, resulting in normal routing actions like stripping data-link headers, making a routing decision, and so on.
Alternately, the Layer 3 switch configuration can make a physical port act like a router interface instead of a switch interface. To do so, the switch configuration makes that port a routed port. On a routed port, the switch does not perform Layer 2 switching logic on that frame. Instead, frames arriving in a routed port trigger the Layer 3 routing logic, including
Stripping off the incoming frame’s Ethernet data-link header/trailer
Making a Layer 3 forwarding decision by comparing the destination IP address to the IP routing table
Adding a new Ethernet data-link header/trailer to the packet
Forwarding the packet, encapsulated in a new frame
This third major section of the chapter examines routed interfaces as configured on Cisco Layer 3 switches, but with a particular goal in mind: to also discuss Layer 3 EtherChannels. The exam topics do not mention routed interfaces specifically, but the exam topics do mention L3 EtherChannels, meaning Layer 3 EtherChannels.
You might recall that Chapter 10, “RSTP and EtherChannel Configuration,” discussed Layer 2 EtherChannels. Like Layer 2 EtherChannels, Layer 3 EtherChannels also treat multiple links as one link. Unlike Layer 2 EtherChannels, however, Layer 3 EtherChannels treat the channel as a routed port instead of switched port. So this section first looks at routed ports on Cisco Layer 3 switches and then discusses Layer 3 EtherChannels.
Implementing Routed Interfaces on Switches
When a Layer 3 switch needs a Layer 3 interface connected to a subnet, and only one physical interface connects to that subnet, the network engineer can choose to use a routed port instead of an SVI. Conversely, when the Layer 3 switch needs a Layer 3 interface connected to a subnet, and many physical interfaces on the switch connect to that subnet, an SVI needs to be used. (SVIs forward traffic internally into the VLAN, so that then the Layer 2 logic can forward the frame out any of the ports in the VLAN. Routed ports cannot.)
To see why, consider the design in Figure 17-4, which repeats the same design from Figure 17-3 (used in the SVI examples). In that design, the gray rectangle on the right represents the switch and its internals. On the right of the switch, at least two access ports sit in both VLAN 10 and VLAN 20. However, that figure shows a single link from the switch to Router B1. The switch could configure the port as an access port in a separate VLAN, as shown with VLAN 30 in Examples 17-6 and 17-7. However, with only one switch port needed, the switch could configure that link as a routed port, as shown in the figure.
){kind=link}
)
FIGURE 17-4 Routing on a Routed Interface on a Switch
Enabling a switch interface to be a routed interface instead of a switched interface is simple: just use the no switchport subcommand on the physical interface. Cisco switches capable of being a Layer 3 switch use a default of the switchport command to each switch physical interface. Think about the word switchport for a moment. With that term, Cisco tells the switch to treat the port like it is a port on a switch—that is, a Layer 2 port on a switch. To make the port stop acting like a switch port and instead act like a router port, use the no switchport command on the interface.
Once the port is acting as a routed port, think of it like a router interface. That is, configure the IP address on the physical port, as implied in Figure 17-4. Example 17-10 shows a completed configuration for the interfaces configured on the switch in Figure 17-4. Note that the design uses the exact same IP subnets as the example that showed SVI configuration in Example 17-6, but now, the port connected to subnet 10.1.30.0 has been converted to a routed port. All you have to do is add the no switchport command to the physical interface and configure the IP address on the physical interface.
Example 17-10 Configuring Interface G0/1 on Switch SW1 as a Routed Port
ip routing ! interface vlan 10 ip address 10.1.10.1 255.255.255.0 ! interface vlan 20 ip address 10.1.20.1 255.255.255.0 ! interface gigabitethernet 0/1 no switchport ip address 10.1.30.1 255.255.255.0
Once configured, the routed interface will show up differently in command output in the switch. In particular, for an interface configured as a routed port with an IP address, like interface GigabitEthernet0/1 in the previous example:
show interfaces: Similar to the same command on a router, the output will display the IP address of the interface. (Conversely, for switch ports, this command does not list an IP address.)
show interfaces status: Under the “VLAN” heading, instead of listing the access VLAN or the word trunk, the output lists the word routed, meaning that it is a routed port.
show ip route: Lists the routed port as an outgoing interface in routes.
show interfaces type number switchport: If a routed port, the output is short and confirms that the port is not a switch port. (If the port is a Layer 2 port, this command lists many configuration and status details.)
Example 17-11 shows samples of all four of these commands as taken from the switch as configured in Example 17-10.
Example 17-11 Verification Commands for Routed Ports on Switches
SW11# show interfaces g0/1
GigabitEthernet0/1 is up, line protocol is up (connected)
Hardware is Gigabit Ethernet, address is bcc4.938b.e541 (bia bcc4.938b.e541)
Internet address is 10.1.30.1/24
! lines omitted for brevity
SW1# show interfaces status
! Only ports related to the example are shown; the command lists physical only
Port Name Status Vlan Duplex Speed Type
Fa0/1 connected 10 a-full a-100 10/100BaseTX
Fa0/2 notconnect 10 auto auto 10/100BaseTX
Fa0/3 connected 20 a-full a-100 10/100BaseTX
Fa0/4 connected 20 a-full a-100 10/100BaseTX
Gi0/1 connected routed a-full a-1000 10/100/1000BaseTX
SW1# show ip route
! legend omitted for brevity
10.0.0.0/8 is variably subnetted, 6 subnets, 2 masks
C 10.1.10.0/24 is directly connected, Vlan10
L 10.1.10.1/32 is directly connected, Vlan10
C 10.1.20.0/24 is directly connected, Vlan20
L 10.1.20.1/32 is directly connected, Vlan20
C 10.1.30.0/24 is directly connected, GigabitEthernet0/1
L 10.1.30.1/32 is directly connected, GigabitEthernet0/1
SW1# show interfaces g0/1 switchport
Name: Gi0/1
Switchport: Disabled
So, with two options—SVI and routed ports—where should you use each?
For any topologies with a point-to-point link between two devices that do routing, a routed interface works well.
Figure 17-5 shows an example of where to use SVIs and where to use routed ports in a typical core/distribution/access design. In this design, the core (Core1, Core2) and distribution (D11 through D14) switches perform Layer 3 switching. All the ports that are links directly between the Layer 3 switches can be routed interfaces. For VLANs for which many interfaces (access and trunk) connect to the VLAN, SVIs make sense because the SVIs can send and receive traffic out multiple ports on the same switch. In this design, all the ports on Core1 and Core2 will be routed ports, while the four distribution switches will use some routed ports and some SVIs.
)
FIGURE 17-5 Using Routed Interfaces for Core and Distribution Layer 3 Links
Implementing Layer 3 EtherChannels
So far, this section has stated that routed interfaces can be used with a single point-to-point link between pairs of Layer 3 switches, or between a Layer 3 switch and a router. However, in most designs, the network engineers use at least two links between each pair of distribution and core switches, as shown in Figure 17-6.
)
FIGURE 17-6 Two Links Between Each Distribution and Core Switch
While each individual port in the distribution and core could be treated as a separate routed port, it is better to combine each pair of parallel links into a Layer 3 EtherChannel. Without using EtherChannel, you can still make each port on each switch in the center of the figure be a routed port. It works. However, once you enable a routing protocol but don’t use EtherChannels, each Layer 3 switch will now learn two IP routes with the same neighboring switch as the next hop—one route over one link, another route over the other link.
Using a Layer 3 EtherChannel makes more sense with multiple parallel links between two switches. By doing so, each pair of links acts as one Layer 3 link. So, each pair of switches has one routing protocol neighbor relationship with the neighbor, and not two. Each switch learns one route per destination per pair of links, and not two. IOS then balances the traffic, often with better balancing than the balancing that occurs with the use of multiple IP routes to the same subnet. Overall, the Layer 3 EtherChannel approach works much better than leaving each link as a separate routed port and using Layer 3 balancing.
Compared to what you have already learned, configuring a Layer 3 EtherChannel takes only a little more work. Chapter 10 already showed you how to configure an EtherChannel. This chapter has already shown how to make a port a Layer 3 routed port. Next, you have to combine the two ideas by combining both the EtherChannel and routed port configuration. The following checklist shows the steps, assuming a static definition.
Step 1. Configure the physical interfaces as follows, in interface configuration mode:
A. Add the channel-group number mode on command to add it to the channel. Use the same number for all physical interfaces on the same switch, but the number used (the channel-group number) can differ on the two neighboring switches.
B. Add the no switchport command to make each physical port a routed port.
Step 2. Configure the PortChannel interface:
A. Use the interface port-channel number command to move to port-channel configuration mode for the same channel number configured on the physical interfaces.
B. Add the no switchport command to make sure that the port-channel interface acts as a routed port. (IOS may have already added this command.)
C. Use the ip address address mask command to configure the address and mask.
Example 17-12 shows an example of the configuration for a Layer 3 EtherChannel for switch SW1 in Figure 17-7. The EtherChannel defines port-channel interface 12 and uses subnet 10.1.12.0/24.
)
FIGURE 17-7 Design Used in EtherChannel Configuration Examples
Example 17-12 Layer 3 EtherChannel Configuration on Switch SW1
interface GigabitEthernet1/0/13 no switchport no ip address channel-group 12 mode on ! interface GigabitEthernet1/0/14 no switchport no ip address channel-group 12 mode on ! interface Port-channel12 no switchport ip address 10.1.12.1 255.255.255.0
Of particular importance, note that although the physical interfaces and PortChannel interface are all routed ports, the IP address should be placed on the PortChannel interface only. In fact, when the no switchport command is configured on an interface, IOS adds the no ip address command to the interface. Then configure the IP address on the PortChannel interface only.
Once configured, the PortChannel interface appears in several commands, as shown in Example 17-13. The commands that list IP addresses and routes refer to the PortChannel interface. Also, note that the show interfaces status command lists the fact that the physical ports and the port-channel 12 interface are all routed ports.
Example 17-13 Verification Commands Listing Interface Port-Channel 12 from Switch SW1
SW1# show interfaces port-channel 12
Port-channel12 is up, line protocol is up (connected)
Hardware is EtherChannel, address is bcc4.938b.e543 (bia bcc4.938b.e543)
Internet address is 10.1.12.1/24
! lines omitted for brevity
SW1# show interfaces status
! Only ports related to the example are shown.
Port Name Status Vlan Duplex Speed Type
Gi1/0/13 connected routed a-full a-1000 10/100/1000BaseTX
Gi1/0/14 connected routed a-full a-1000 10/100/1000BaseTX
Po12 connected routed a-full a-1000
SW1# show ip route
! legend omitted for brevity
10.0.0.0/8 is variably subnetted, 4 subnets, 2 masks
C 10.1.2.0/24 is directly connected, Vlan2
L 10.1.2.1/32 is directly connected, Vlan2
C 10.1.12.0/24 is directly connected, Port-channel12
L 10.1.12.1/32 is directly connected, Port-channel12
For a final bit of verification, you can examine the EtherChannel directly with the show etherchannel summary command as listed in Example 17-14. Note in particular that it lists a flag legend for characters that identify key operational states, such as whether a port is bundled (included) in the PortChannel (P) and whether it is acting as a routed (R) or switched (S) port.
Example 17-14 Verifying the EtherChannel
SW1# show etherchannel 12 summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator
M - not in use, minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
Number of channel-groups in use: 1
Number of aggregators: 1
Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
12 Po12(RU) - Gi1/0/13(P) Gi1/0/14(P)
Troubleshooting Layer 3 EtherChannels
When you are troubleshooting a Layer 3 EtherChannel, there are two main areas to consider. First, you need to look at the configuration of the channel-group command, which enables an interface for an EtherChannel. Second, you should check a list of settings that must match on the interfaces for a Layer 3 EtherChannel to work correctly.
As for the channel-group interface subcommand, this command can enable EtherChannel statically or dynamically. If dynamic, this command’s keywords imply either Port Aggregation Protocol (PaGP) or Link Aggregation Control Protocol (LACP) as the protocol to negotiate between the neighboring switches whether they put the link into the EtherChannel.
If all this sounds vaguely familiar, it is the exact same configuration covered way back in the Chapter 10 section “Configuring Dynamic EtherChannels.” The configuration of the channel-group subcommand is exactly the same, with the same requirements, whether configuring Layer 2 or Layer 3 EtherChannels. So, it might be a good time to review those EtherChannel configuration details from Chapter 10. However, regardless of when you review and master those commands, note that the configuration of the EtherChannel (with the channel-group subcommand) is the same, whether Layer 2 or Layer 3.
Additionally, you must do more than just configure the channel-group command correctly for all the physical ports to be bundled into the EtherChannel. Layer 2 EtherChannels have a longer list of requirements, but Layer 3 EtherChannels also require a few consistency checks between the ports before they can be added to the EtherChannel. The following is the list of requirements for Layer 3 EtherChannels:
no switchport: The PortChannel interface must be configured with the no switchport command, and so must the physical interfaces. If a physical interface is not also configured with the no switchport command, it will not become operational in the EtherChannel.
Speed: The physical ports in the channel must use the same speed.
duplex: The physical ports in the channel must use the same duplex.