Shooting Trouble with IP

As more and more companies are adopting IP-based networks, the jobs of network engineers and analysts are becoming more demanding. Learn about IP protocols and packets, addressing, and routing protocol topics to get you started with IP troubleshooting.

This chapter focuses on a number of objectives falling under the CCNP Troubleshooting guidelines. Understanding basic TCP/IP troubleshooting principles not only applies to the CCNP certification but to all industry certifications. A solid understanding of how IP works is essential for troubleshooting any small, medium, or large network.

This chapter and the remaining chapters assume knowledge of the previous chapters, which deal conceptually with protocol characteristics, models, troubleshooting methods, support tools, and resources. Each chapter starts by introducing a hands-on chapter scenario. To gain practical experience, build the network in the scenario if at all possible and follow along. If that isn't possible, the content and explanations are detailed enough for you to learn from without needing the equipment in front of you. Several integrated walk-through scenarios and Trouble Tickets enable you to benefit from the added learning advantages offered by practical application. After the Shooting Trouble with IP scenario, I explore TCP/IP concepts, symptoms, problems, and action plans.

This chapter covers the following topics:

• Scenario: Shooting Trouble with IP
• Protocols and Packets
• Addressing
• Routing Protocols
• Trouble Tickets
• Trouble Tickets Solutions

Scenario: Shooting Trouble with IP

It is now time to get started with the practical Shooting Trouble with IP scenario. First, add the additional equipment, perform a write erase or erase startup-config to clear your configurations from previous labs, and rewire according to Figure 3-1.

Figure 3-1 Shooting Trouble with IP

The scenario goal is to put in the basic configurations, and then add Routing Information Protocol (RIP) as the routing protocol to work toward end-to-end connectivity between the hosts. Where appropriate, use best practices such as descriptions on interfaces, hosts tables, and so on. Configure r1 and work your way through r5. As always, test and document along the way and when you finish configuring.

Remember, however, that there is not always one right or wrong way to accomplish the tasks presented. The ability to obtain the end result using good practices is extremely important in any real-world network. My troubleshooting and device configurations start in Example 3-1; you can compare your work to that and perhaps see a different approach to obtaining the end result. Figure 3-2 shows a picture of my lab before wiring, and Figure 3-3 shows the after-wiring picture. I have physically labeled each of my devices so that I don't have to think about that later. Refer back to Figure 3-1 as you continue to set up and troubleshoot.

Figure 3-2 Scenario Lab Photo Before Wiring

Figure 3-3 Scenario Lab Photo After Wiring

The terminal server at the top of the equipment stack in Figure 3-2 is not a required piece of equipment for the lab, but more a convenience. I am using a Cisco 2511 with the first five terminal leads connected to each of the console ports on my five routers. See Appendix C for more information on how to configure a terminal server and use one for your lab. My 2511 configuration is in Example 3-1 with the significant output shaded.

Example 3-1 Terminal Server Configuration (2511)

ts#show running-config 
...
hostname ts
enable password donna
ip subnet-zero
ip host r1 2001 1.1.1.1
ip host r2 2002 1.1.1.1
ip host r3 2003 1.1.1.1
ip host r4 2004 1.1.1.1
ip host r5 2005 1.1.1.1
interface Loopback0
 ip address 1.1.1.1 255.0.0.0
 no ip directed-broadcast
...
line con 0
 transport input none
line 1 16
 transport input all
 no exec
line aux 0
line vty 0 4
 exec-timeout 30 0
 password donna
 logging synchronous
 login
end

Although I give you Figures 3-1 through 3-3, it is really a better practice to draw your own network diagram. Some people prefer columns and rows of this type of data, but I prefer colorful diagrams to assist with troubleshooting later. For example, you might draw your devices and media with a blue pen, label the IP parameters with a black pen, label IPX parameters with a red pen, draw a green circle around the Open Shortest Path First (OSPF) areas, and so on. Label which interfaces are DCE or DTE for your lab. Document device names, locations, Layer 2 and Layer 3 addresses, routed and bridged protocols, routing protocols, access control lists (ACLs), configuration files, and verify full connectivity. Perform some simple ping and trace tests (see Table 3-1), run show tech-support, and document some more. All of this gives you a starting point for normal baseline activity when your network is running well. Keep in mind that I want you to concentrate only on IP-related baselining for this chapter.

Table 3-1 gives you a layered yet divide-and-conquer approach to quickly spotting IP issues. It would be wonderful if I could tell you to just start at the first item in the table and work your way through, but you need to think methodically (as Chapter 1, "Shooting Trouble," suggested). It is helpful to divide and conquer along the way in practical application to quickly narrow down the real problem. If you can't communicate with your gateway, for instance, it is a little difficult to communicate with a remote host. If you can't communicate with yourself, it is impossible to communicate with a local host.

Table 3-1 IP Troubleshooting Checklist

Isolating Problems	Commands and Symptoms
Check MAC address, IP address, subnet mask, default gateway, and other static or DHCP1 parameters.	Windows NT/2000: ipconfig /all Windows 95/98: winipcfg UNIX: ifconfig
Ping your loopback from your workstation to see whether the TCP/IP stack is loaded.	ping 127.0.0.1
Ping yourself from your workstation to verify your NIC2.	ping 192.168.1.11
Ping a local host from your workstation to verify local communications.	ping 192.168.1.12
Ping your default gateway from your workstation to verify you can communicate with your local router interface.	ping 192.168.1.1
Are you getting ARPs3 from the gateway? If so, the gateway's MAC address should be in the workstation ARP table.	arp –a
Ping a remote host and another if it fails from your workstation.	ping 192.168.3.5
Perform a trace to the remote host to find hop-by-hop router issues.	Windows: tracert [-d] UNIX/Cisco: traceroute
Is it a host problem or a router problem?	show ip interface brief show run interface e0 show ip interface e0 show interfaces e0 show ip route show ip protocols show ip arp show ip cache show ip access-list
Combine ping and trace to look for packet loss in the path.	pathping 192.168.3.5
Use an application to test the upper layers. NetBIOS issues Sockets issues	Start => Run \\192.168.3.5 Find computer nbtstat –A IP_address ping 192.168.3.1 telnet 192.168.3.1 ftp 192.168.3.1 tftp 192.168.3.1
Eliminate any name resolution issues by not using hostnames or NetBIOS names at first. When other things are working, fix any name resolution issues. Ensure name resolution files are in their required locations and have the appropriate names. Troubleshoot files, DNS4/WINS5 servers, and the network issues to and from these devices.	Name resolution DNS, hosts file, NIS tables WINS, lmhosts file (NetBIOS) nbtstat –c (view cache) nbtstat –R (reload cache) Microsoft Browser services (NetBIOS issues) Start, Run \\computername Find computer or net view nbtstat –A IP_address Sockets issues ping hostc telnet hostc

Using the scenario diagram in Figure 3-1, configure r1 similar to what is in Example 3-2. My r1 is a Cisco 2514, but you can use any Cisco router that has two Ethernet interfaces and two serial interfaces for the lab. My passwords are all donna because that is easy to remember for the labs, but that is exactly why they should not all be donna for practical application. Throughout the following examples, I have made a few careless mistakes that you may or may not make. I will troubleshoot them when all my routers are configured per the scenario diagram.

Example 3-2 r1 Configuration (2514)

Router>enable
Router#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)#hostname r1
r1(config)#enable password donna
r1(config)#line vty 0 4
r1(config-line)#login
r1(config-line)#password donna
r1(config-line)#exit
r1(config)#interface ethernet 0
r1(config-if)#description e0 to hosta and hostb
r1(config-if)#ip address 192.168.1.1 255.255.255.0
r1(config-if)#no shut
00:10:12: %LINK-3-UPDOWN: Interface Ethernet0, changed state to up 
00:10:13: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0,
  changed state to up
r1(config)#interface ethernet 1
r1(config-if)#description e1 to r2e0 
r1(config-if)#ip address 192.168.4.1 255.255.255.0
r1(config-if)#no shut
r1(config-if)#interface serial 0
r1(config-if)#description s0 to r5s0
r1(config-if)#ip address 10.1.1.1 255.255.255.0
r1(config-if)#bandwidth 64
r1(config-if)#no shut
00:13:11: %LINK-3-UPDOWN: Interface Serial0, changed state to down
r1(config-if)#ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
r1(config)#ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
r1(config)#$192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
r1(config)#ip host r4 10.2.2.2
r1(config)#ip host r5 10.1.1.2
r1(config)#router rip
r1(config-router)#network 192.168.1.0
r1(config-router)#network 192.168.2.0
r1(config-router)#network 192.168.4.0
r1(config-router)#network 10.1.1.0
r1(config-router)#end
r1#copy running-config startup-config

Now move on to configuring r2 as in Example 3-3. My r2 is a Cisco 2501, but you can use any Cisco router that has at least one Ethernet interface and two serial interfaces for the lab. I copied the hosts table lines from r1 and pasted them into this configuration. In future examples, I plan to just paste the configuration for the hosts table and passwords to save a little typing.

Example 3-3 r2 Configuration (2501)

Router(config)#hostname r2
r2(config)#enable password donna
r2(config)#line vty 0 4
r2(config-line)#login
r2(config-line)#password donna
r2(config-line)#exit
r2(config)#interface ethernet 0
r2(config-if)#description e0 to r1e1
r2(config-if)#ip address 192.168.4.2 255.255.255.0
r2(config-if)#no shut
r2(config-if)#int 
00:41:44: %LINK-3-UPDOWN: Interface Ethernet0, changed state to up
00:41:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0,
  changed state to ups0
r2(config-if)#description s0 to r3s0/1
r2(config-if)#bandwidth 64
r2(config-if)#ip address 192.168.6.1 255.255.255.0
r2(config-if)#no shut
00:42:22: %LINK-3-UPDOWN: Interface Serial0, changed state to down
r2(config-if)#interface serial 1
r2(config-if)#description s1 to r3s0/2
r2(config-if)#bandwidth 64
r2(config-if)#ip address 192.168.5.1 255.255.255.0
r2(config-if)#router rip
r2(config-router)#network 192.168.4.0
r2(config-router)#network 192.168.5.0
r2(config-router)#network 192.168.6.0
r2(config-router)#exit
r2(config)#ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
r2(config)#ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
r2(config)#$192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
r2(config)#ip host r4 10.2.2.2
r2(config)#ip host r5 10.1.1.2
r2(config)#end
r2#copy running-config startup-config

Configure the rest of your routers now and check your work using the following examples. I copied the text in Example 3-4 to Windows Notepad to easily paste it into r3, r4, and r5.

Example 3-4 Notepad File Including Passwords and Hosts Table

enable password donna
ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
ip host r3 192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
ip host r4 10.2.2.2
ip host r5 10.1.1.2
line vty 0 4
login
password donna
line console 0
logging synchronous
exit

Example 3-5 and Example 3-6 start my r3 configuration. My r3 is a Cisco 3640, but you can use any Cisco router that has at least one Ethernet interface and four serial interfaces for the lab. Although the capabilities are not important in this chapter, having multiple serial interfaces on a router enables you to set up your own Frame Relay switch later in the book. Depending on the capabilities, the Fast Ethernet interface will give you an opportunity to experiment with duplex and speed concepts as well.

Note in Example 3-5 that I attempted to configure the e0 interface when it was really fa2/0 that I needed to configure. A physical inspection of the device confirmed that the Fast Ethernet port was located in Slot 2; because you can't physically see my device, however, I proceeded with the show interfaces command.

Example 3-5 r3 Configuration (3640)

Router(config)#hostname r3
r3(config)#enable password donna
r3(config)#ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
r3(config)#ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
r3(config)#$192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
r3(config)#ip host r4 10.2.2.2
r3(config)#ip host r5 10.1.1.2
r3(config)#line vty 0 4
r3(config-line)#login
r3(config-line)#password donna
r3(config-line)#line console 0
r3(config-line)#logging synchronous
r3(config-line)#exit
r3(config)#int e0
        ^
% Invalid input detected at '^' marker.
r3(config)#end
r3#show interfaces
Serial0/0 is administratively down, line protocol is down 
 Hardware is CD2430 in sync mode
...
FastEthernet2/0 is administratively down, line protocol is down 
 Hardware is AmdFE, address is 00b0.6481.e300 (bia 00b0.6481.e300)
 MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec, rely 255/255, load 1/255
 Encapsulation ARPA, loopback not set, keepalive set (10 sec)
 Half-duplex, 100Mb/s, 100BaseTX/FX
 ARP type: ARPA, ARP Timeout 04:00:00
...
r3#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
r3(config)#interface fastethernet 2/0
r3(config-if)#ip address 192.168.3.1 255.255.255.0
r3(config-router)#interface serial 0/0
r3(config-if)#desc r3s0/0 to r1s1
r3(config-if)#bandwidth 64
r3(config-if)#clock rate 64000
r3(config-if)#ip address 192.168.2.2 255.255.255.0
r3(config-if)#no shut
r3(config-if)#interface serial 0/1
r3(config-if)#description r3s0/1 to r2s0
r3(config-if)#bandwidth 64
r3(config-if)#clock rate 64000
r3(config-if)#ip address 192.168.6.2 255.255.255.0
r3(config-if)#no shut
r3(config-if)#interface serial 0/2
r3(config-if)#description r3s0/2 to r2s1
r3(config-if)#bandwidth 64
r3(config-if)#clock rate 64000
r3(config-if)#ip address 192.168.5.2 255.255.255.0
r3(config-if)#no shut

Finish configuring r3, r4, and r5 and test your configurations.

Now that you have configured your lab, perform some basic lower-layer tests to verify your drawing and your internetwork. Make sure all used interfaces are in a line protocol up state as in Example 3-6; if they are not in a line protocol up state, fix any noticeable problems at this point. Notice how show ip interface brief is a very appropriate command to quickly spot lower-level issues.

Example 3-6 IP Interface Testing

r1>show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Ethernet0       192.168.1.1   YES NVRAM  up            up   
Ethernet1       192.168.4.1   YES manual up            up   
Serial0         10.1.1.1      YES NVRAM  up            up   
Serial1        unassigned   YES unset administratively down down  
r2>show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Ethernet0       192.168.4.2   YES NVRAM   up          up   
Serial0         192.168.6.1   YES NVRAM   up          up   
Serial1         192.168.5.1   YES NVRAM administratively down down  
r3>show ip interface brief
Interface       IP-Address       OK? Method Status        Protocol
Serial0/0       192.168.2.2      YES unset  down          down  
Serial0/1       192.168.6.2      YES unset  up            up   
Serial0/2       192.168.5.2      YES unset  down          down  
Serial0/3       unassigned       YES unset  down          down  
...
FastEthernet2/0    192.168.3.1   YES manual up            down  
r4>show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Ethernet0/0      unassigned   YES unset administratively down down  
Serial0/0       10.2.2.2    YES manual down         down  
Serial0/1       unassigned   YES unset administratively down down  
r5>sh ip int brie 
Interface       IP-Address   OK? Method Status           Protocol
BRI0          unassigned     YES unset  administratively down down  
BRI0:1         unassigned    YES unset  administratively down down  
BRI0:2         unassigned    YES unset  administratively down down  
Ethernet0       unassigned   YES unset  administratively down down  
Serial0        10.1.1.2      YES manual up          up   
Serial1        unassigned    YES unset  administratively down down

Think about these line and protocol issues. Target the lower layers to get all the required interfaces to a status of up/up before you continue. Check your work using the following examples.

First I spotted, for interface s1, the unassigned IP address and the administratively down status on r1, which I correct in Example 3-7. Because my interface command was interrupted once more, I must have forgotten logging synchronous on r1, so I added it and saved the configuration.

Example 3-7 Correcting Interface Issues on r1

r1(config)#interface serial 1
r1(config-if)#description s1 to r3s0/0
r1(config-if)#bandwidth 64
r1(config-if)#ip address 192.168.2.1 255.255.255.0
r1(config-if)#no shut
05:16:39: %LINK-3-UPDOWN: Interface Serial1, changed state to up
05:16:40: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial1, changed state  to up
r1(config-if)#end
r1#sh i
05:16:50: %SYS-5-CONFIG_I: Configured from console by consolep int brief
Interface       IP-Address    OK? Method Status        Protocol
Ethernet0       192.168.1.1   YES NVRAM up             up   
Ethernet1       192.168.4.1   YES manual up            up   
Serial0         10.1.1.1      YES NVRAM up             up   
Serial1         192.168.2.1   YES manual up            up   
r1#configure terminal 
Enter configuration commands, one per line. End with CNTL/Z.
r1(config)#line console 0
r1(config-line)#logging synchronous 
r1(config-line)#end
r1#copy running-config startup-config

The r1s1 interface would have come up fine without the bandwidth statement, but it is optimal for routing protocols to configure the correct bandwidth statement on your interfaces. The description is optional as well, but it certainly makes troubleshooting easier when you know exactly what is connected to an interface. Now move along to r2, which has issues with interface s1 being administratively down. Fix these issues now and check your work in Example 3-8.

Example 3-8 Correcting Interface Issues on r2

r2(config)#interface serial 1
r2(config-if)#no shut
05:20:08: %LINK-3-UPDOWN: Interface Serial1, changed state to up
05:20:09: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial1,
  changed state to up
r2(config-if)#end
r2#show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Ethernet0       192.168.4.2   YES NVRAM up            up   
Serial0         192.168.6.1   YES NVRAM up            up   
Serial1         192.168.5.1   YES NVRAM up            up   
r2#copy running-config startup-config

r3 requires you to look at your drawing more closely so that you can concentrate on just the interfaces being used. Configure any missing IP addresses and issue a no shut command on any used interfaces that are showing as administratively down. Check the status of the interfaces in Example 3-9.

Example 3-9 Correcting Interface Issues on r3

r3#show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Serial0/0       192.168.2.2   YES manual up           up   
Serial0/1       192.168.6.2   YES manual up           up   
Serial0/2       192.168.5.2   YES manual up           up   
Serial0/3       unassigned    YES manual down         down  
...
FastEthernet2/0    192.168.3.1   YES manual up          down

Example 3-9 indicates that a problem still exists with s0/3 and fa2/0. The other end (host) is not running for my Ethernet hostc connection, but you need to examine further the cause of the down/down status for s0/3. Think about what's in your tool bag from the preceding chapter to assist you further in spotting lower-layer problems. Check your thoughts against Example 3-10.

Example 3-10 Correcting Physical Issues on r3

r3#show controllers serial 0/3
CD2430 Slot 0, Port 3, Controller 0, Channel 3, Revision 15
Channel mode is synchronous serial 
idb 0x6129A1A0, buffer size 1524, V.35 DTE cable
...

Everything looks normal on the r3 end of things from a physical point of view, so now investigate the other end of the connection as in Example 3-11.

Example 3-11 Investigate r4 serial 0/0 Connection

r4#show controllers serial 0/0
Interface Serial0/0
Hardware is Quicc 68360
No serial cable attached
idb at 0x60AC9A40, driver data structure at 0x60ACEE10
...

I have an advantage in that I can physically inspect my devices; I hope you can do the same if you are following along in your own lab. If you look very closely in the picture of my equipment, you may be able to detect the error, but I won't assume that for now. The show controllers commands certainly display the problem here. Although I did not specifically illustrate the output of show controllers s0/1, the output of s0/0 is quite helpful. I had the cable plugged into s0/1 rather than s0/0 on r4. On the 3640, s0/0 is closest to the power switch, which is typical. This mistake affected the serial connection between r3 and r4. Example 3-12 shows the output show ip interface brief after the physical correction and assigning the appropriate address to s0/3.

Example 3-12 After the Physical Cable Swap from serial 0/1 to serial 0/0

r3#show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Serial0/0       192.168.2.2   YES manual up          up   
Serial0/1       192.168.6.2   YES manual up          up   
Serial0/2       192.168.5.2   YES manual up          up   
Serial0/3       10.2.2.1      YES manual up          up   
...
FastEthernet2/0 192.168.3.1   YES manual up          down  
r3#copy running-config startup-config

After you bring your hosts back online, the Fast Ethernet 2/0 status should change from up/down to up/up. I give that a test in Example 3-13. My Fast Ethernet interface did not come up when I brought the host online, so follow along to determine the issue.

The first thing I noted was that the network card dongle did not light up for 10 or 100 Mbps. Next, look at Figure 3-1 and label what type of cable you need if you have the PC connected directly into the Fast Ethernet port. Category 5 crossover is correct. I fixed the problem by replacing my original straight-through cable with a crossover Category 5 in-line coupler so that I could use two short straight-through cables to make my connection. Figure 3-4 shows a picture of the coupler. In practical application, this is where using colored cables would help you to very quickly spot the issue. For example, use the normal gray cable for straight-through and use red for crossovers. Things appear to be working for now in Example 3-13.

Figure 3-4 Crossover Category 5 In-line Coupler

Example 3-13 Fast Ethernet 2/0 Status

r3#show run interface fastethernet 2/0
interface FastEthernet2/0
 ip address 192.168.3.1 255.255.255.0
 no ip directed-broadcast
end
...
07:22:02: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet2/0,
  changed state to up
r3#show ip interface brief
Interface       IP-Address   OK?  Method Status        Protocol
Serial0/0       192.168.2.2   YES manual up            up   
Serial0/1       192.168.6.2   YES manual up            up   
Serial0/2       192.168.5.2   YES manual up            up   
Serial0/3       10.2.2.1      YES manual up            up   
...
FastEthernet2/0 192.168.3.1   YES manual up            up

Make sure you have made all corrections, including those that you need for your lab, so that you can continue the tests in Example 3-14 for some simple router ping tests. Recall from the preceding chapters that ping tests up through Layer 3.

Example 3-14 Testing the Scenario with Ping

r1>ping r2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.4.2, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 4/4/4 ms
r1>ping r3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.2.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 28/30/32 ms
r1>ping r4
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.2.2.2, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)
r1>ping r5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 28/30/32 ms

Next check the routing tables and routing protocols as in Example 3-15 to make sure r1 has a route to get to r4.

Example 3-15 r1 Routing Table

r1>show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
    i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default
    U - per-user static route, o - ODR
Gateway of last resort is not set...
C  192.168.4.0/24 is directly connected, Ethernet1
R  192.168.5.0/24 [120/1] via 192.168.2.2, 00:00:02, Serial1
   10.0.0.0/24 is subnetted, 1 subnets
C    10.1.1.0 is directly connected, Serial0
R  192.168.6.0/24 [120/1] via 192.168.2.2, 00:00:02, Serial1
C  192.168.1.0/24 is directly connected, Ethernet0
C  192.168.2.0/24 is directly connected, Serial1
R  192.168.3.0/24 [120/1] via 192.168.2.2, 00:00:02, Serial1
r1>show ip protocols 
Routing Protocol is "rip"
 Sending updates every 30 seconds, next due in 3 seconds
 Invalid after 180 seconds, hold down 180, flushed after 240
 Outgoing update filter list for all interfaces is not set 
 Incoming update filter list for all interfaces is not set
 Redistributing: rip
 Default version control: send version 1, receive any version
  Interface    Send Recv  Key-chain
  Ethernet0    1   1 2          
  Ethernet1    1   1 2          
  Serial0     1   1 2          
  Serial1     1   1 2          
 Routing for Networks:
  10.0.0.0
  192.168.1.0
  192.168.2.0
  192.168.4.0
 Routing Information Sources:
  Gateway     Distance   Last Update
  192.168.2.2     120   00:00:02
 Distance: (default is 120)

Continue to think about the issue here; the output contains some pretty useful information (particularly the shaded areas). However, you should analyze any problems that I specifically mentioned and fix them now. Check your configurations against mine so that you can return and continue to test out end-to-end host connectivity. I made a few other minor changes, which I highlight in the next few examples. Examples 3-16 through 3-21 include the running configurations for all my routers at this time.

Example 3-16 r1 (2514) Configuration

r1#show running-config 
...
hostname r1
enable password donna
ip subnet-zero
ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
ip host r3 192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
ip host r4 10.2.2.2
ip host r5 10.1.1.2
!
interface Ethernet0
 description e0 to hosta and hostb
 ip address 192.168.1.1 255.255.255.0
 no ip directed-broadcast
interface Ethernet1
 description e1 to r2e0 
 ip address 192.168.4.1 255.255.255.0
 no ip directed-broadcast
interface Serial0
 description s0 to r5s0
 bandwidth 64
 ip address 10.1.1.1 255.255.255.0
 no ip directed-broadcast
 no ip mroute-cache
 no fair-queue
interface Serial1
 description s1 to r3s0/0
 bandwidth 64
 ip address 192.168.2.1 255.255.255.0
 no ip directed-broadcast
router rip
 network 10.0.0.0
 network 192.168.1.0
 network 192.168.2.0
 network 192.168.4.0
ip classless
line con 0
 logging synchronous
 transport input none
line aux 0
line vty 0 4
 password donna
 login
end
r1#

Next look at r2's configuration in Example 3-17.

Example 3-17 r2 (2501) Configuration

r2#show running-config 
...
hostname r2
enable password donna
ip subnet-zero
ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
ip host r3 192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
ip host r4 10.2.2.2
ip host r5 10.1.1.2
interface Ethernet0
 description e0 to r1e1
 ip address 192.168.4.2 255.255.255.0
 no ip directed-broadcast
interface Serial0
 description s0 to r3s0/1
 bandwidth 64
 ip address 192.168.6.1 255.255.255.0
 no ip directed-broadcast
 no ip mroute-cache
 no fair-queue
interface Serial1
 description s1 to r3s0/2
 bandwidth 64
 ip address 192.168.5.1 255.255.255.0
 no ip directed-broadcast
router rip
 network 192.168.4.0
 network 192.168.5.0
 network 192.168.6.0
ip classless
line con 0
 logging synchronous
 transport input none
line aux 0
line vty 0 4
 password donna
 login
end
r2#

Make any adjustments to your r2, and then analyze the r3 configuration in Example 3-18.

Example 3-18 r3 (3640) Configuration

r3#show running-config 
...
hostname r3
enable password donna
ip subnet-zero
ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
ip host r3 192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
ip host r4 10.2.2.2
ip host r5 10.1.1.2
interface Serial0/0
 description s0/0 to r1s1
 bandwidth 64
 ip address 192.168.2.2 255.255.255.0
 no ip directed-broadcast
 no ip mroute-cache
 clockrate 64000
interface Serial0/1
 description s0/1 to r2s0
 bandwidth 64
 ip address 192.168.6.2 255.255.255.0
 no ip directed-broadcast
 clockrate 64000
interface Serial0/2
 description s0/2 to r2s1
 bandwidth 64
 ip address 192.168.5.2 255.255.255.0
 no ip directed-broadcast
 clockrate 64000
...
interface FastEthernet2/0
 description fa2/0 to hostc
 ip address 192.168.3.1 255.255.255.0
 no ip directed-broadcast
router rip
 network 10.0.0.0
 network 192.168.2.0
 network 192.168.3.0
 network 192.168.5.0
 network 192.168.6.0
ip classless
line con 0
 logging synchronous
 transport input none
line aux 0
line vty 0 4
 password donna
 login    
end
r3#

Descriptions are the only modifications I found necessary and this may seem a little mundane, but documentation is extremely helpful for troubleshooting. Next analyze the r4 configuration in Example 3-19.

Example 3-19 r4 (3620) Configuration

r4#show running-config 
...
hostname r4
enable password donna
ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
ip host r3 192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
ip host r4 10.2.2.2
ip host r5 10.1.1.2
interface Ethernet0/0
 no ip address
 shutdown    
interface Serial0/0
 description s0/0 to r3s0/3
 ip address 10.2.2.2 255.255.255.0
 no ip mroute-cache
 bandwidth 64
 clockrate 64000
interface Serial0/1
 no ip address
 shutdown
router rip
 network 10.0.0.0
ip classless
line con 0
 logging synchronous
line aux 0
line vty 0 4
 password donna
 login
end    
r4#

Last but not least, compare your r5 configuration to Example 3-20. \

Example 3-20 3-20 r5 (2516) Configuration

r5#show running-config 
...
hostname r5
enable password donna
ip subnet-zero
ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
ip host r3 192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
ip host r4 10.2.2.2
ip host r5 10.1.1.2
hub ether 0 1
 link-test
 auto-polarity
...
interface Serial0
 description s0 to r1s0
 bandwidth 64
 ip address 10.1.1.2 255.255.255.0
 no ip directed-broadcast
 no ip mroute-cache
 no fair-queue
 clockrate 64000
...
router rip
 network 10.0.0.0
ip classless
line con 0
 logging synchronous
 transport input none
line aux 0
line vty 0 4
 password donna
 login    
end
r5#

Although you have tested many things, there is still a problem with the routing table display (as you witnessed in Example 3-15). There are many ways to fix this problem, and by now I predict that you thoroughly looked through the configurations and researched your network diagram (refer to Figure 3-1) to spot the issue of discontiguous subnets.

The Routing Information Protocol (RIP) does not support discontiguous subnets, or at least RIPv1 does not support this. However, what would be the result of changing the routing protocol to RIPv2? Example 3-21 displays the steps necessary to change the existing RIPv1 to RIPv2 on r1; repeat these steps on r2 through r5 as well.

Example 3-21 Configuring RIPv2

r1(config)#router rip
r1(config-router)#version ?
 <1-2> version
r1(config-router)#version 2
r1(config-router)#end
r1#copy running-config startup-config

Test and analyze the results of your configuration. First look at the output of show ip route and show ip protocols as in Example 3-22 and test with ping as in Example 3-23. Display the routing table, the IP routing protocols, and ensure that r1 can ping all other routers before you continue. Compare the results to Figure 3-1 to determine if anything is missing.

Example 3-22 Testing and Analyzing r1 RIPv2 Routes and Protocols

r1#show ip route
...
C  192.168.4.0/24 is directly connected, Ethernet1
R  192.168.5.0/24 [120/1] via 192.168.2.2, 00:00:13, Serial1
   10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
R    10.0.0.0/8 [120/1] via 192.168.2.2, 00:00:13, Serial1
C    10.1.1.0/24 is directly connected, Serial0
R  192.168.6.0/24 [120/1] via 192.168.2.2, 00:00:13, Serial1
C  192.168.1.0/24 is directly connected, Ethernet0
C  192.168.2.0/24 is directly connected, Serial1
R  192.168.3.0/24 [120/1] via 192.168.2.2, 00:00:13, Serial1
r1#show ip protocols 
Routing Protocol is "rip"
 Sending updates every 30 seconds, next due in 1 seconds
 Invalid after 180 seconds, hold down 180, flushed after 240
 Outgoing update filter list for all interfaces is not set
 Incoming update filter list for all interfaces is not set
 Redistributing: rip
 Default version control: send version 2, receive version 2
  Interface    Send Recv  Key-chain
  Ethernet0    2    2           
  Ethernet1    2    2           
  Serial0      2    2           
  Serial1      2    2           
 Routing for Networks:
  10.0.0.0
  192.168.1.0
  192.168.2.0
  192.168.4.0
 Routing Information Sources:
  Gateway     Distance   Last Update
  192.168.2.2     120   00:00:05
 Distance: (default is 120)
r2#show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
    i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default
    U - per-user static route, o - ODR
Gateway of last resort is not set
C  192.168.4.0/24 is directly connected, Ethernet0
C  192.168.5.0/24 is directly connected, Serial1
R  10.0.0.0/8 [120/1] via 192.168.4.1, 00:00:05, Ethernet0
        [120/1] via 192.168.6.2, 00:00:16, Serial0
        [120/1] via 192.168.5.2, 00:00:16, Serial1
C  192.168.6.0/24 is directly connected, Serial0
R  192.168.1.0/24 [120/1] via 192.168.4.1, 00:00:05, Ethernet0
R  192.168.2.0/24 [120/1] via 192.168.4.1, 00:00:05, Ethernet0
          [120/1] via 192.168.6.2, 00:00:16, Serial0
          [120/1] via 192.168.5.2, 00:00:16, Serial1
R  192.168.3.0/24 [120/1] via 192.168.6.2, 00:00:16, Serial0
          [120/1] via 192.168.5.2, 00:00:16, Serial1

Perform your ping tests if you like, but you may not be any more successful than you were with RIPv1 from r2's standpoint. Even if you were, for a hint as to the real issue here, the same thing would occur if I told you to use Enhanced Interior Gateway Routing Protocol (EIGRP) instead of RIPv2. The fix is in Example 3-23, so check your thoughts against it and make any necessary changes to your configurations.

Example 3-23 Turning Off Automatic Summarization and Reviewing the Routing Table

r1(config)#router rip
r1(config-router)#no auto-summary 
r1(config-router)#end
r1#copy running-config startup-config
r3(config)#router rip
r3(config-router)#no auto-summary 
r3(config-router)#end
r3#copy running-config startup-config
r2#show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
    i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default
    U - per-user static route, o - ODR
Gateway of last resort is not set
C  192.168.4.0/24 is directly connected, Ethernet0
C  192.168.5.0/24 is directly connected, Serial1
   10.0.0.0/24 is subnetted, 2 subnets
R    10.2.2.0 [120/1] via 192.168.6.2, 00:00:00, Serial0
         [120/1] via 192.168.5.2, 00:00:00, Serial1
R    10.1.1.0 [120/1] via 192.168.4.1, 00:00:20, Ethernet0
C  192.168.6.0/24 is directly connected, Serial0
R  192.168.1.0/24 [120/1] via 192.168.4.1, 00:00:20, Ethernet0
R  192.168.2.0/24 [120/1] via 192.168.6.2, 00:00:00, Serial0
          [120/1] via 192.168.5.2, 00:00:00, Serial1
          [120/1] via 192.168.4.1, 00:00:20, Ethernet0
R  192.168.3.0/24 [120/1] via 192.168.6.2, 00:00:01, Serial0
          [120/1] via 192.168.5.2, 00:00:01, Serial1

As a result of the no auto-summary command on r1 and r3, Example 3-23 clearly illustrates that r2 is less confused about where to send the packets destined for network 10.0.0.0.

RIPv2 is classless and RIPv1 is classful. Classless routing protocols—such as RIPv2, EIGRP, OSPF, and Intermediate System-to-Intermediate System (IS-IS)—support variable-length subnet masking (VLSM) and summarization. All routing protocols support summarization, but the classful ones do that in a fixed manner, at the class boundary. Hence in this example, the routers could not differentiate between 10.1.1.0/24 and 10.2.2.0/24 because a completely different network separated them. This is what is meant by discontiguous subnets. The "Routing Protocol" and "Summarization" sections of this chapter provide a little more detail. You can count on gaining more practical experience with this issue in the Trouble Tickets.

After you verify router connectivity, move on to verify host-to-host connectivity as in Example 3-24. You may find Table 3-1 to be a helpful guide here. This is a good time to check the interfaces and routing tables on all your other routers, too. Although I didn't display the output, all of my routers can ping all my other routers using the configured hosts table.

Example 3-24 displays the host-to-host testing from hosta to hostc. I assume you took time to configure the appropriate default gateways for your hosts; if not, do that now.

Example 3-24 Testing and Analyzing Host-to-Host Connectivity from hosta to hostc

C:\>ipconfig
Windows 2000 IP Configuration
Ethernet adapter Local Area Connection:
    Connection-specific DNS Suffix . :
    IP Address. . . . . . . . . . . . : 192.168.1.11
    Subnet Mask . . . . . . . . . . . : 255.255.255.0
    Default Gateway . . . . . . . . . : 192.168.1.1
!!!hosta can ping itself
C:\>ping 192.168.1.11
Pinging 192.168.1.11 with 32 bytes of data:
Reply from 192.168.1.11: bytes=32 time<10ms TTL=128
Reply from 192.168.1.11: bytes=32 time<10ms TTL=128
Reply from 192.168.1.11: bytes=32 time<10ms TTL=128
Reply from 192.168.1.11: bytes=32 time<10ms TTL=128
Ping statistics for 192.168.1.11:
  Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
  Minimum = 0ms, Maximum = 0ms, Average = 0ms
!!!hosta can ping its gateway
C:\>ping 192.168.1.1
Pinging 192.168.1.1 with 32 bytes of data:
Reply from 192.168.1.1: bytes=32 time<10ms TTL=255
Reply from 192.168.1.1: bytes=32 time<10ms TTL=255
Reply from 192.168.1.1: bytes=32 time<10ms TTL=255
Reply from 192.168.1.1: bytes=32 time<10ms TTL=255
Ping statistics for 192.168.1.1:
  Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
  Minimum = 0ms, Maximum = 0ms, Average = 0ms
!!!hosta can ping hostc
C:\>ping 192.168.3.5
Pinging 192.168.3.5 with 32 bytes of data:
Request timed out.
Reply from 192.168.3.5: bytes=32 time=20ms TTL=126
Reply from 192.168.3.5: bytes=32 time=10ms TTL=126
Reply from 192.168.3.5: bytes=32 time=10ms TTL=126
Ping statistics for 192.168.3.5:
  Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),
Approximate round trip times in milli-seconds:
  Minimum = 10ms, Maximum = 20ms, Average = 10ms
C:\>

Just in case you have any issues, you might find it helpful to display the host routing tables as in Example 3-25. Hosta has a default gateway of its local router interface 192.168.1.1. Hostc has a default gateway of 192.168.3.1, which you can verify by issuing the route print or ipconfig/winipcfg commands on the host.

Example 3-25 Testing and Analyzing Host-to-Host Connectivity from hosta Continued

C:\>route print
Interface List
0x1 ........................... MS TCP Loopback interface
0x2 ...44 45 53 54 42 00 ...... NOC Extranet Access Adapter
0x1000004 ...00 10 4b a5 ae 50 ...... FE575 Ethernet Adapter
Active Routes:
Network         Destination     Netmask      Gateway       Interface Metric
0.0.0.0         0.0.0.0         192.168.1.1  192.168.1.11    1
127.0.0.0       255.0.0.0       127.0.0.1    127.0.0.1       1
192.168.1.0     255.255.255.0   192.168.1.11 192.168.1.11    1
192.168.1.11    255.255.255.255 127.0.0.1    127.0.0.1       1
192.168.1.255   255.255.255.255 192.168.1.11 192.168.1.11    1
224.0.0.0       224.0.0.0       192.168.1.11 192.168.1.11    1
255.255.255.255 255.255.255.255 192.168.1.11 2               1
Default Gateway:    192.168.1.1
Persistent Routes:
 None

Now that you have a working IP scenario, it's time to investigate some of the underlying components of the TCP/IP suite.

Protocols and Packets

This section covers some of the protocols, applications, and utilities at each layer of the TCP/IP suite that may assist you with supporting day-to-day internetworks. First the Internet Layer is discussed, then the Transport Layer, and finally the Application Layers. Each layered subsection contains protocol analysis and packet captures, including a review of the packet headers, to help you better understand the packet traces and prepare you for troubleshooting on your own.

Table 3-2 and the following subsections provide a layered perspective of many of the TCP/IP-related protocols, applications, and utilities.

Table 3-2 TCP/IP Protocols, Applications, and Utilities

Layer	ISO's OSI Model	DoD TCP/IP Suite	Protocols, Applications, and Utilities
7	Application	Application	Telnet, NFS, FTP, TFTP, HTTP, DNS, X.500, *RIP, *BGP, *DHCP, ASCII, EBCDIC, JPEG, GIF, NetBIOS, SOCKETS
6	Presentation
5	Session
4	Transport	Transport Host-to-Host	TCP, UDP, *OSPF, *EIGRP
3	Network	Internet	IP, ICMP, ARP/RARP
2	Data Link	Data Link	Ethernet, Token Ring, FDDI, Frame Relay, ATM, ISDN, HDLC, over various media types
1	Physical	Physical

Frame Types

Encapsulation, frame format, frame type—they all mean the same thing, which is packaging the upper-layer data, voice, or video into an Layer 2 frame. See Chapter 4, "Shooting Trouble with Novell IPX," and the detailed Layer 2 LAN and WAN chapters for frame type information. Part III of this book covers supporting Ethernet, switches, and virtual LANs (VLANs) and Part IV is about supporting the WAN.

Internet Layer Protocols, Applications, and Utilities

Internet protocols such as those listed in the Table 3-2 are well suited for LAN and WAN heterogeneous communications. The Internet suite of protocols includes not only TCP and IP but also many upper-layer applications and utilities for file, print, messaging, database, and other common practical services.

Ensure that your lab is up and running properly so that you spend your efforts on what happens behind the scenes when hosta pings hostc from an IP standpoint. In this section, you turn on your Sniffer on segment 1 and at least capture a ping from hosta to hostc so that you can walk through the fields in the IP header.

Example 3-26 lists in bold the recommended steps to perform from hosta while capturing the packets with Sniffer Pro or the protocol analyzer you are using for your lab.

Example 3-26 Steps Performed on hosta While Capturing the Packets with Sniffer Pro

C:\>ping 192.168.3.5
Pinging 192.168.3.5 with 32 bytes of data:
Request timed out.
Reply from 192.168.3.5: bytes=32 time=20ms TTL=126
Reply from 192.168.3.5: bytes=32 time=10ms TTL=126
Reply from 192.168.3.5: bytes=32 time=10ms TTL=126
Ping statistics for 192.168.3.5:
  Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),
Approximate round trip times in milli-seconds:
  Minimum = 10ms, Maximum = 20ms, Average = 10ms
C:\>ping 192.168.3.5
Pinging 192.168.3.5 with 32 bytes of data:
Reply from 192.168.3.5: bytes=32 time=10ms TTL=126
Reply from 192.168.3.5: bytes=32 time=10ms TTL=126
Reply from 192.168.3.5: bytes=32 time=10ms TTL=126
Reply from 192.168.3.5: bytes=32 time=20ms TTL=126
Ping statistics for 192.168.3.5:
  Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
  Minimum = 10ms, Maximum = 20ms, Average = 12ms
C:\>tracert 192.168.3.5
Tracing route to HOSTC [192.168.3.5]
over a maximum of 30 hops:
 1  10 ms  <10 ms  <10 ms 192.168.1.1
 2  20 ms  20 ms  20 ms 192.168.2.2
 3  20 ms  30 ms  30 ms HOSTC [192.168.3.5]
Trace complete.
C:\>pathping 192.168.3.5
Tracing route to HOSTC [192.168.3.5]
over a maximum of 30 hops:
 0 HOSTA [192.168.1.11]
 1 192.168.1.1
 2 192.168.2.2
 3 HOSTC [192.168.3.5]
Computing statistics for 75 seconds...
      Source to Here  This Node/Link
Hop RTT  Lost/Sent = Pct Lost/Sent = Pct Address
 0                      HOSTA [192.168.1.11]
                0/ 100 = 0%  |
 1  0ms   0/ 100 = 0%   0/ 100 = 0% 192.168.1.1
                0/ 100 = 0%  |
 2  20ms   0/ 100 = 0%   0/ 100 = 0% 192.168.2.2
                0/ 100 = 0%  |
 3  20ms   0/ 100 = 0%   0/ 100 = 0% HOSTC [192.168.3.5]
Trace complete.
C:\>arp -a
Interface: 192.168.1.11 on Interface 0x1000005
 Internet Address   Physical Address   Type
 192.168.1.1      00-00-0c-8d-67-05   dynamic
C:\>

Table 3-3 lists the fields of the IP header (packet or datagram), and Figure 3-5 shows the first significant ping lines of the Sniffer packet capture of hosta pinging hostc. Refer to the file chapter 3 ping from hosta to hostc sniffer capture.

Table 3-3 The IP Header (Packet or Datagram)

Fields	Bits	Description
Version	4	Version of IP.
Header length	4	Length in 32-bit words (HLEN).
Priority and TOS	8 (more detail)	Datagram handling for the upper-layer protocol (delay, throughput, reliability, and cost).
Total length	16	Maximum length of datagram is 65535 bytes (data and header).
Fields	Bits	Description
Identification	16	Identifies smaller fragments that need to be re-assembled back into the same packet.
Flags	3	Specify whether packet can be fragmented and whether there are any more fragments.
Fragment offset	13	The order (byte count) of the fragment in the big packet for re-assembly purposes.
TTL	8	Time-to-live for the packet. When it decrements to 0, the packet is discarded. Keeps packets from looping forever.
Protocol	8	Pointer to the upper-layer protocol.
Header checksum	16	For header integrity.
Source IP address		32-bit sending node.
Destination IP address	32	32-bit receiving node.
Options	0–32	Allows IP to support such options as security, testing, or debugging.
Data	Varies	The actual data and upper-layer information.

Figure 3-5 Hosta ARPs the Router

View the summary pane to be aware of the general packet flow when hosta initiated a ping to hostc in Figure 3-5. Hosta is on network 192.168.1.0/24, and hostc is on network 192.168.3.0/24, so hosta relied on its default gateway (local router interface) in which to hand the packets. Line 28 shows the local ARP request as a broadcast command, and line 29 shows the unicast ARP response from 192.168.1.1 (default gateway). Compare this to the arp –a output on hosta back in Example 3-26. Lines 30 to 37 illustrate the ICMP Echo Requests and Echo Replies. Normally, Microsoft hosts issue four requests and four replies, whereas Cisco routers give you five by default. However, there are not four replies here. Look back at Example 3-26 to analyze why.

Move along to the Echo Request packet on line 32 in Figure 3-6. Notice how I sized my windows to see more of the detailed IP header. These are the same fields that are in Table 3-3. Compare them one-by-one until you are comfortable with the IP packet structure (although this will not be your last chance to do this). Also note the differences between the Echo Request and the Echo Reply packet. Figure 3-7 displays the output of an Echo Reply.

Figure 3-6 Analyzing the IP Header of an Echo Request Packet

Figure 3-7 Analyzing the IP Header of an Echo Reply Packet

The main differences you should have noted in the Sniffer traces include the following:

• The source and destination MAC addresses are reversed per the Data Link Control (DLC) header.

• The source and destination IP addresses are reversed per the IP header.

• The Internet Control Message Protocol (ICMP) type is 8 for the Echo Request and 0 for the Echo Reply per the ICMP header.

Real-world packet analysis should make you a little more comfortable with how IP works, but I'll certainly test that out in the upcoming Trouble Tickets. IP is the main protocol at the Internet Layer that has helpers such as ARP and ICMP to assist it with its duties. ARP is for resolving an IP address to a MAC address, whereas Reverse Address Resolution Protocol (RARP) is for resolving a MAC address to an IP address. ICMP is for status and error reporting. Look back at Example 3-26 to see the status lines, such as Reply from..., and the error reporting, such as Request timed out. Obviously, this is why you didn't have four successful replies in the protocol analyzer capture. Also note in the Example that 192.168.3.5 is nowhere to be found in the ARP cache; instead, the default gateway IP and MAC address is there. Example 3-27 displays the e0 interface on r1 so that you can compare this information. Also note that r1 has hosta 192.168.1.11 in its IP and ARP cache.

Example 3-27 r1 Ethernet 0 IP and MAC Addresses

r1#show interfaces ethernet 0
Ethernet0 is up, line protocol is up 
 Hardware is Lance, address is 0000.0c8d.6705 (bia 0000.0c8d.6705)
 Description: e0 to hosta and hostb
 Internet address is 192.168.1.1/24
 MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec, rely 255/255, load 1/255
 Encapsulation ARPA, loopback not set, keepalive set (10 sec)
 ARP type: ARPA, ARP Timeout 04:00:00
 Last input 00:00:00, output 00:00:09, output hang never
 Last clearing of "show interface" counters never
Queueing strategy: fifo
 Output queue 0/40, 0 drops; input queue 1/75, 0 drops
 5 minute input rate 0 bits/sec, 0 packets/sec
 5 minute output rate 0 bits/sec, 0 packets/sec
   2358 packets input, 361385 bytes, 0 no buffer
   Received 1949 broadcasts, 0 runts, 0 giants, 0 throttles
   0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
   0 input packets with dribble condition detected
   2889 packets output, 320787 bytes, 0 underruns
   0 output errors, 11 collisions, 2 interface resets
   0 babbles, 0 late collision, 15 deferred
   0 lost carrier, 0 no carrier
   0 output buffer failures, 0 output buffers swapped out
r1#show ip cache
IP routing cache 1 entry, 172 bytes
  5 adds, 4 invalidates, 0 refcounts
Minimum invalidation interval 2 seconds, maximum interval 5 seconds,
  quiet interval 3 seconds, threshold 0 requests
Invalidation rate 0 in last second, 0 in last 3 seconds
Prefix/Length     Age    Interface    Next Hop
192.168.1.11/32     03:02:54 Ethernet0    192.168.1.11
r1#show arp
Protocol Address           Age (min) Hardware Addr  Type  Interface
Internet 192.168.1.11      34        0010.4ba5.ae50 ARPA  Ethernet0
Internet 192.168.1.1       -         0000.0c8d.6705 ARPA  Ethernet0
Internet 192.168.4.1       -         0000.0c8d.6706 ARPA  Ethernet1
Internet 192.168.4.2       42        0000.0c38.a05d ARPA  Ethernet1
r1#

I want to continue the layered approach to discussing TCP/IP, so next the discussion moves up the stack to the Transport Layer. After I discuss the Transport and Application Layers, I spend a bit more time on addressing and protocols before venturing into the Trouble Tickets.

Transport (Host-to-Host) Layer Protocols, Applications, and Utilities

Recall from the OSI model in Chapter 1, the Transport Layer is all about host-to-host delivery. TCP and UDP are the Transport Layer twins. TCP is connection-oriented (logical connection) and reliable (ACKs).

UDP, like IP at the Internet Layer, is connectionless and unreliable; therefore it relies on the upper layers for reliability. TCP is like the certified mail protocol, whereas UDP is like the regular mail (or better yet, bulk mail) protocol.

UDP is connectionless, which means it does not require an established connection before communications can occur. It is unreliable at the Transport Layer, which means that its reliability is left up to the application. Compare the TCP and UDP packet formats in Tables 3-4 and 3-5.

Table 3-4 The TCP Segment

Fields	Bits	Description
Source Port	16	Sending port.
Destination Port	16	Receiving port.
Sequence Number	32	Tracks byte transfer.
Acknowledgment Number	32	Confirms byte transfer. Forward referenced and expectational in that it contains the sequence number of the next byte expected.
Data Offset	4	Number of 32-bit words in the header.
Reserved	6	For future use.
Flags	6 flags of TCP	Synchronization (SYN) Acknowledgement (ACK) Finish (FIN) Push (PSH) Urgent (URG) Reset (RST)
Window Size	16	How many bytes are sent per segment. Size of sender's receive window. Incoming buffer space. (How many pizzas can you put in the warming bag if you are the delivery person?)
Checksum		Sender generates and receiver verifies to see whether the header was damaged in transit.
Urgent Pointer		Points to first urgent data byte, such as Ctrl+Z to end urgent data.
Options	32	Various options must end on a 32-bit boundary, and padding guarantees this.
Data	Varies	Upper-layer information.

Table 3-5 The UDP Datagram

Fields	Bits	Description
Source Port	16	Sending port
Destination Port	16	Receiving port
UDP Length	16	Data and header
Checksum	16	Optional
Data	Varies	Upper-layer information

The 20-byte TCP header is a lot more sophisticated than the 8-byte UDP header, as you will again confirm with Sniffer. First, open the previous Sniffer Pro trace where you issued the ping and tracert commands in Example 3-26. I named my file chapter 3 ping from hosta to hostc sniffer capture. Use this file to analyze the layers or refer to Figure 3-8, in which I emphasize the layers of a RIP packet.

Although the focus in this subsection is the Transport Layer, remember that each layer depends on the functionality of another layer to carry out communications. The Sniffer trace certainly illustrates that. For example, the RIP packet at the Data Link Layer is a good example of multicast communications. RIPv2 works via multicast address 01005E000009 at Layer 2, whereas RIPv1 works via broadcast, which would be shown as all Fs at Layer 2. The EtherType is 0800, which tells the Data Link Layer to hand off to IP at the Internet Layer for further processing.

Review the IP header fields—Version, Header length, TOS, Total length, Identification, Flags, Fragment offset, TTL, Protocol, Checksum, Addresses, and Options. The IP header contains the Protocol field to hand off to at the next layer in the stack. In Figure 3-8, for example, the protocol number is 17, which means to hand up to UDP. The source address is the Ethernet 0 interface on r1, and the destination is again the reserved multicast address of 224.0.0.9 for RIPv2, but this time at the Internet Layer.

IANA has reserved addresses from 224.0.0.0 through 224.0.0.255, such as those in Table 3-6, for local multicasts.

Figure 3-8 Analyzing the RIP Packet at Layers 2, 3, and 4

Table 3-6 Local Multicast Addresses

Multicast Address	Description
224.0.0.1	All systems on this subnet
224.0.0.2	All routers on this subnet
224.0.0.5	OSPF routers
224.0.0.6	OSPF designated routers
224.0.0.9	RIPv2

Globally scoped addresses from 224.0.1.0 through 238.255.255.255 can be used to multicast data between organizations and across the Internet. An example of an IANA reserved address is 224.0.1.1 for Network Time Protocol (NTP).

In addition, IANA owns a block of Ethernet MAC addresses that start with 01:00:5E, where half of the block (0100.5e00.0000 through 0100.5e7f.ffff) is allocated for multicast addresses. In the Ethernet address, 23 bits correspond to the IP multicast group address. Search for "ethernet mac multicast" at Cisco.com to get a detailed explanation and pictures. With this mapping, the upper 5 bits of the IP multicast address are dropped and the resulting address is not unique, which results in different multicast group IDs that all map to the same Ethernet address. The Internet Group Management Protocol (IGMP) dynamically registers individual hosts in a multicast group. The hosts send IGMP messages to their router. The routers listen and periodically send out queries to discover which groups are active or inactive on a particular subnet.

The Transport Layer contains the fields listed Table 3-5 for the UDP datagram. RIP works via UDP port 520, which is clearly revealed here. UDP port 520 is how the Transport Layer hands off to the Application Layer for RIP communications (as discussed in more detail later). For now, look at the details of TCP.

TCP provides end-to-end full-duplex delivery, flow control through windowing, and error-detection and -correction services. Data moves in a continuous byte stream, in which bytes are identified by sequence numbers. TCP hides lower-layer intricacies from the upper layers on the receiving host. It segments and re-assembles data for upper-layer applications based on various port numbers. Unlike UDP, however, a 3-way handshake must occur before communications can begin. This establishes the virtual connection between the two communicating parties (see Figure 3-9).

Figure 3-9 The TCP 3-Way Handshake Sequence

A good example of an application that uses TCP is telnet. Try it out and telnet from hosta (at the command prompt) to r1. Capture your results with Sniffer (see Figure 3-10). Save the Sniffer file as chapter 3 telnet from hosta to r1 sniffer capture so that you can refer back to it later. Analyze your own capture or look at my Sniffer trace. It may prove helpful to use the Sniffer output to label the 3-way handshake in Figure 3-9, including flags, ports, sequence, and acknowledgement numbers.

Figure 3-10 Analyzing Telnet and the 3-Way Handshake

In the preceding example, hosta was configured with the local router IP address as its default gateway. Because of this, hosta sent an ARP request packet to its default gateway to learn the MAC address of the Ethernet 0 interface on r1. The command arp –a on the host would have shown this, whereas show ip arp is the command on the router. ARP frames are not part of the 3-way handshake or TCP session, but are certainly required for hosta to transmit data.

Study TCP in the Sniffer capture and drawing. It is often referred to as the 3-way handshake. Step 1 of the 3-way handshake (SYN) is like me introducing myself to you and giving you my basic communication parameters so that we can talk. Step 2 is like you saying, "Okay (ACK), Donna, I want to talk, too; here are my communication parameters (SYN)." Step 3 is my okay (ACK) to you. After a 3-way handshake, the two communicating parties are virtually connected and TCP communications can then occur. Some applications require multiple handshakes. A bona fide example is anything involving the World Wide Web (WWW). Every time you click a link on a web page, another TCP session starts. Another example is a phone call. The connection is set up and you talk; then the logical connection is torn down and is available for someone else.

Next look a little closer at the exact packets in the Sniffer capture for the 3-way handshake. You can glean a lot from the summary pane on this one, but the detail pane is shown as well. The shaded line 7 starts the handshake described in Figure 3-9. It shows the SYN from hosta (192.168.1.11) to the router (192.168.1.1). The source port (S) is random (ephemeral) port number 1079, but the destination port is the well-known port number 23 for telnet.

Ports are places to leave stuff for applications to pick up, as you will continue to see throughout this book. A client typically establishes a port within first 4 bytes of the Transport Layer header. Notice how the ports reverse depending on who is doing the talking, hosta or the router. Line 8 shows the router responding back to hosta with ACK number ...2608, which if you look close is one more than the previous SEQ number of ...2607. The SEQ of Step 2 of the handshake is ...3396. Step 3 of the handshake in line 9 ACKs the previous SEQ number with ...3397. When the TCP session has been established, the numbers increment by the actual number of bytes transferred.

Instead of analyzing the telnet details, I want you to pay attention to the TCP session tear down next. Open your Sniffer capture and refer to the very end of the file as I illustrate in Figure 3-11.

Figure 3-11 TCP Session Disconnect

Frame 132 starts the TCP disconnect with the FIN flag from the router. Hosta ACKs in frame 133 and says, "Hey, I am also finished," in frame 134 (FIN). The router gives the final ACK in the last frame. A 4-way disconnect like this is common.

Next I want you to think outside the box a little. What would have happened if the default gateway was not configured on the host or if the router was not local in the preceding testing? You can give me the number one phrase that most support people give, which is, it depends; as usual, however, I'll cringe at that answer. In many cases, however, that is the best answer. Actually, the packets may still get to their destination assuming proxy ARP is enabled on the router and that hosta will ARP for nonlocal destinations.

Proxy ARP helps hosts reach remote subnets without configuring routing or a default gateway. Configuring the host with a smaller subnet mask would make the host ARP for everything and thus send all packets via the router. The router just replies to the host with its MAC address assuming that it is configured to accept and respond to proxy ARP. Obviously, there are security and overhead disadvantages to proxy ARP. The Cisco IOS interface command no ip proxy-arp turns this off and is shown in Example 3-28.

Example 3-28 Proxy ARP

r1#show run interface ethernet 0
interface Ethernet0
 description e0 to hosta and hostb
 ip address 192.168.1.1 255.255.255.0
 no ip directed-broadcast
end
r1#configure terminal
r1(config)#interface ethernet 0
r1(config-if)#no ip proxy-arp
r1(config-if)#end
r1#show run interface ethernet 0 
interface Ethernet0
 description e0 to hosta and hostb
 ip address 192.168.1.1 255.255.255.0
 no ip directed-broadcast
 no ip proxy-arp
end
r1#configure terminal
r1(config)#interface e0
r1(config-if)#ip proxy-arp 
r1(config-if)#end

Prior to the thinking outside the box exercise, you were working with telnet, which is an application based on TCP port 23. TCP port 23 is the pointer from the Transport Layer to the Application Layer, just as the IP packet contains a protocol number such as 6 to link to the Transport Layer TCP protocol for further processing. Figure 3-12 shows this layer linkage for the UDP and TCP applications you previously examined. You can always compare this to your saved Sniffer traces to validate the theory from the lower layers to the upper layers.

Figure 3-12 Protocols, Applications, and Utilities

Many applications have well-known port numbers assigned. Ports greater than 1024 are referred to as ephemeral, random, short-lived, or temporary, whereas numbers below 1024 are considered well-known ports.

Figure 3-13, http://www.iana.org, and RFC 1700 provide you with more detail. Ports are often categorized as follows:

• 0–1023 Well-known

• 1024–49151 Registered

• 49152–65535 Dynamic (private)

Figure 3-13 Application Port Numbers

As you have seen in the previous subsections, after you have eliminated Physical Layer issues, protocol connections are troubleshooting targets that must be considered. After the protocol connections have been confirmed as operational, it is time to move up to the Application Layers. I could not begin to cover the vast variety of upper-layer applications in use today, but I will introduce some of the major Application Layer protocols of the TCP/IP suite.

Upper-Layer Protocols, Applications, and Utilities

This subsection covers applications such as telnet, FTP, TFTP, SMTP, POP3, DNS, SNMP, RIP, HTTP, HTTPS, and DHCP. Obviously, these are not the only applications you will need to troubleshoot, but they are very common.

First look at the terminal emulation protocol telnet from an application perspective. Review Figures 3-9 and 3-10 and your Sniffer capture file (chapter 3 telnet from hosta to r1 sniffer capture). Figure 3-14 illustrates the telnet session from hosta on port 1079 to r1 on port 23. Line 10 starts the telnet session, which was dependent on the TCP 3-way handshake in lines 7, 8, and 9. Notice the echo, window size, and terminal negotiation in lines 10 to 15. Frame 12 is waiting for the login that eventually appears in lines 17 to 25 one character at a time. In addition, notice how that after the user-level password I typed the necessary command and password to get into enable mode. Although I do not show all this in the screen capture of Figure 3-14, if you have your own file you can see the rest of the commands that were typed on the router, letter for letter. It is pretty obvious here that the standard telnet programs do not encrypt the login information, and I doubt that this is what you want people to see when you telnet to your devices to configure them. In the real-world application of telnet where security is more of a concern, many people use secure telnet programs (Secure Shell [SSH] port 22). Examples include SecureCRT, CommNet, and PuTTY.SSH.

Figure 3-14 Telnet from an Application Perspective

Next look at TFTP and FTP. As I illustrated in Figure 3-13, TFTP typically operates over UDP port 69, and FTP typically operates over TCP port 21 for control and 20 for data. When I say typically, I really mean that this is entirely up to the developer. FTP and TFTP are very useful applications in the support environment. For instance, what happens if you lose the configuration on one of your routers? Hopefully, you have an automated way to restore it instead of having to type in the configuration line-for-line.

To test TFTP, I am using a freeware program called PumpKin in Figure 3-15. You can download PumpKin, set up another router as a TFTP server, or use any TFTP application you like. I started and configured PumpKin to put and get all files, started a new Sniffer capture, and proceeded to copy my r1 configuration file to the TFTP server for this test. You should even go a few steps further to wipe your configuration with the erase startup-config or write erase command to ensure this really works. After all, this is the type of thing you should do in a lab environment many times so that you are prepared for the unexpected. Use Example 3-29 as a guide for this exercise.

Save the Sniffer file as chapter 3 tftp from r1 to hosta and back sniffer capture for further review.

Figure 3-15 Using PumpKin for a TFTP Server

Example 3-29 Copying r1 Configuration to a TFTP Server

 r1#copy running-config tftp
Address or name of remote host []? 192.168.1.11
Destination filename [running-config]? hosta-config
!!
1093 bytes copied in 5.888 secs (218 bytes/sec)
r1#erase startup-config
Erasing the nvram filesystem will remove all files! Continue? [confirm]
[OK]
Erase of nvram: complete
!!!The next 2 shaded commands are not necessary for the tftp exercise
!!!However, it is a good time to review when files are replaced or 
!!!just modified.
r1#copy startup-config running-config
Destination filename [running-config]? 
r1#show running-config
Building configuration...
Current configuration:
version 12.0
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
hostname r1
enable password donna
ip subnet-zero
ip host r1 192.168.1.1 192.168.2.1 192.168.4.1 10.1.1.1
ip host r2 192.168.4.2 192.168.5.1 192.168.6.1
ip host r3 192.168.2.2 192.168.5.2 192.168.6.2 192.168.3.1 10.2.2.1
ip host r4 10.2.2.2
ip host r5 10.1.1.2
...!

Next reload the router as in Example 3-30.

Example 3-30 Reloading the Router

r1#reload
Proceed with reload? [confirm]
02:21:13: %SYS-5-RELOAD: Reload requested
System Bootstrap, Version 5.2(8a), RELEASE SOFTWARE
Copyright 1986-1995 by cisco Systems
2500 processor with 2048 Kbytes of main memory
F3: 7464832+102636+503004 at 0x3000060
       Restricted Rights Legend
...
Cisco Internetwork Operating System Software 
IOS (tm) 2500 Software (C2500-IS-L), Version 12.0(5), RELEASE SOFTWARE (fc1)
Copyright 1986-1999 by cisco Systems, Inc.
Compiled Tue 15-Jun-99 19:57 by phanguye
Image text-base: 0x0303D744, data-base: 0x00001000
cisco 2500 (68030) processor (revision L) with 2048K/2048K bytes of memory.
Processor board ID 03074719, with hardware revision 00000000
Bridging software.
X.25 software, Version 3.0.0.
2 Ethernet/IEEE 802.3 interface(s)
2 Serial network interface(s)
32K bytes of non-volatile configuration memory.
8192K bytes of processor board System flash (Read ONLY)
%Error opening tftp://255.255.255.255/network-confg (Timed out)
%Error opening tftp://255.255.255.255/cisconet.cfg (Timed out)
SETUP: new interface Ethernet0 placed in "shutdown" state
SETUP: new interface Ethernet1 placed in "shutdown" state
%Error opening tftp://255.255.255.255/network-confg (Timed out)
%Error opening tftp://255.255.255.255/cisconet.cfg (Timed out)
%Error opening tftp://255.255.255.255/router-confg (Timed out)
%Error opening tftp://255.255.255.255/ciscortr.cfg (Timed out)
Press RETURN to get started!
...
00:03:45: %SYS-5-RESTART: System restarted --
Cisco Internetwork Operating System Software 
IOS (tm) 2500 Software (C2500-IS-L), Version 12.0(5), RELEASE SOFTWARE (fc1)
Copyright 1986-1999 by cisc
Router>o Systems, Inc.
Compiled Tue 15-Jun-99 19:57 by phanguye
Router>enable
Router#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)#interface ethernet 0
Router(config-if)#ip address 192.168.1.1 255.255.255.0
Router(config-if)#no shut
...
Router(config-if)#end

Now copy the file from the TFTP to r1 as in Example 3-31. By the way, a router with no configuration is often referred to as a router out of the box (ROTB).

Example 3-31 Copying the Configuration from the TFTP Server

Router#copy tftp running-config
Address or name of remote host []? 192.168.1.11
Source filename []? hosta-config
Destination filename [running-config]? 
Accessing tftp://192.168.1.11/hosta-config...
Loading hosta-config from 192.168.1.11 (via Ethernet0): !
[OK - 1093/2048 bytes]
1093 bytes copied in 5.124 secs (218 bytes/sec)
r1#copy running-config startup-config
Destination filename [startup-config]? 
Building configuration...

In the preceding example, I copied my file to a TFTP server and made sure it was really there. Then I performed an erase start on the router to erase the startup configuration. When I tried to overwrite the running configuration with the startup configuration, however, it acted as a merge, which is what you should expect. To really get rid of the running configuration, you must reload the router; so I did. When the router came back up, it had no configuration, but it was certainly looking for one (as you can see from the shaded output in Example 3-30). Next I configured the IP address on the Ethernet 0 interface and issued a no shut. If the TFTP server were not local, a default gateway would be required as well. Now that I had TCP/IP communications, I continued and copied the configuration file from the TFTP server back to the router and saved the configuration.

TFTP and FTP can assist you with saving configurations and IOS images depending on your IOS version. To explore the differences, save the TFTP capture or refer to my Sniffer files to later compare it to FTP. Make a list of the major differences as you observe the two applications.

Before you experiment with FTP, take a few minutes to decode the TFTP Sniffer capture. The shaded line in Figure 3-16 highlights a TFTP write request, which Sniffer portrays as opcode 2 in the detail pane. Lines 63 to 69 include the actual file transfer. Line 64 is an opcode 3 and is the first data packet transfer. Notice that the first block, or 512 bytes, of data was transferred and line 65 is the application acknowledgement to the first data packet with an opcode 4. Table 3-7 lists the common opcodes.

Table 3-7 OpCodes

OpCode Text	OpCode Number
Read request	1
Write request	2
Data	3
Ack	4
Error	5

Figure 3-16 TFTP Write Request

Also note that the Sniffer trace does not in any way hide the configuration while it is transferring. Figure 3-17 shows the ending configuration, including passwords. Notice how you can read the entire configuration file when you open each data packet individually.

Using the Sniffer capture, take a few minutes to draw a simple picture of what happens when you transfer a file using TFTP. Include communications between r1 and hosta, including line numbers, ports, and opcodes, as I did in Figure 3-18.

Many UDP-based applications such as some implementations of TFTP and DNS use a fixed data length (such as 512 bytes) to operate, but the maximum per segment is 65535 bytes.

Figure 3-17 TFTP Clear Text

Figure 3-18 TFTP Communications

As you verified in the previous figures, TFTP transfers text in the clear. Use FTP to perform the same file transfer as in the TFTP exercise. Actually, I transferred the configuration from r1 and r2 to the FTP server in my test. Any FTP application is appropriate. I am using 3CDaemon in Figure 3-19, a freeware program I downloaded from support.3com.com/infodeli/swlib/utilities_for_windows_32_bit.htm for the lab in Example 3-32. Use Sniffer to capture the results, analyze your findings, list or draw a picture of what happens when you transfer a file using FTP, and, last but not least, compare the two IP-based applications.

Figure 3-19 FTP Server Software

Example 3-32 Copying r1 and r2 Configuration to an FTP Server

r1(config)#ip ftp username anonymous
r1(config)#ip ftp password donna@shoretraining.com
r1(config)#end
r1#copy running-config startup-config
r1#copy running-config ftp
Address or name of remote host []? 192.168.1.11
Destination filename [running-config]? r1-config
Writing r1-config !
1169 bytes copied in 8.836 secs (146 bytes/sec)
...

I did not take the time to erase my configuration and test this out as I did with the TFTP example. If you need more practice, you can do just that. My chapter 3 ftp from r1 and r2 to hosta sniffer capture file displays in Figure 3-20. Follow the TCP sessions and data transfer, including the sequence and acknowledgement numbers for a review.

Figure 3-20 FTP Write Request

Feel free to draw a more specific picture with line numbers, ports, and sequence and acknowledgement numbers. However, the following gives a general picture of how FTP works:

• ARP and TCP 3-way handshake (FTP control port 21)

• FTP user login (anonymous) and password (donna@shoretraining.com)

• TCP 3-way handshake (FTP data transfer port 20)

• FTP data transfer (clear text)

• Session tear down (data port 20)

• Session tear down (control port 21)

The main difference is that the program I used for the test does not use an individual port 20 for each data transfer, but instead uses a separate ephemeral port.

You should have notes similar to the following to compare TFTP and FTP communications:

• The client uses an ephemeral port to initiate communications to TFTP server port 69 and the server picks an ephemeral port to respond to the client. FTP uses port 21 for control and a separate port 20 for file transfer, and the server port doesn't need to change for multiple clients because of the TCP session.

• No login, username, or password is used for TFTP. FTP requires login and can allow anonymous login with the e-mail address for the password.

• TFTP transfers a minimum of 512 bytes of data per datagram, and the application ACKs each one individually because UDP has no reliability mechanism; FTP, on the other hand, uses a TCP session for each file transfer.

• TFTP and FTP use clear-text data transmission. FTP secure implementations are available.

In addition, TFTP and FTP are very helpful in the Cisco environment to copy not only configurations but also Cisco Operating System images. Many times I set up my router as a TFTP server so that I can copy the IOS to another location. Appendix B provides more detail on such topics.

As you can see, it is helpful to understand the basics of how applications work to more easily troubleshoot them. Next take a look at the Simple Mail Transfer Protocol (SMTP) and Post Office Protocol (POP3) in Figure 3-21.

Figure 3-21 SMTP and POP3

SMTP is like the mail truck delivering from post office to post office. If you work for Cisco Press and you want to set up your mail client to send your mail, for example, you may set up smtp.ciscopress.com, which by default occurs on TCP port 25.

POP3 is more of a server to the client mail delivery protocol that operates by default on TCP port 110. Therefore if you work for Cisco Press and want to set up your mail client, you may set up pop3.ciscopress.com to receive your mail.

Assume user@ciscopress.com wants to send an e-mail to donna@shoretraining.com. That user clicks Send and SMTP is used to transfer the e-mail from the user to his mail server using SMTP TCP port 25. The mail server delivers the mail to the shoretraining.com domain over port 25. When I check my mail on shoretraining, my e-mail program establishes a connection to my mail server over POP3 port 110.

Knowing the overall SMTP/POP3 delivery process may assist you with troubleshooting your e-mail one day. For example, I have been in situations where I could send e-mail but not receive, which turned out to be a POP3 issue. Perhaps a particular port or address was blocked by an access list. I have also been in situations where the opposite occurs. For example, many times you may have the luxury of using a higher-speed network connection but your ISP may not allow you to "relay" through another system. However, troubleshooting e-mail issues happens to be another book in itself (as are most applications).

DNS is another application that it is quite helpful to understand in the support world. I would much rather type http://www.cisco.com than its IP address every time I want to look up something on Cisco.com. DNS maps an IP address to a hostname so that humans can relate to it a little better.

Hostname resolution once was a flat hosts file, but who would have enough memory to open such a thing today? Hosts files are still available for use, however. You have them on each of your routers now. Type show hosts with me on r1 as in Example 3-33.

Example 3-33 Cisco Hosts File

r1>show hosts
Default domain is not set
Name/address lookup uses domain service
Name servers are 255.255.255.255
Host           Flags   Age Type  Address(es)
r1            (perm, OK) 23  IP  192.168.1.1 192.168.2.1
                        192.168.4.1 10.1.1.1
r2            (perm, OK) 23  IP  192.168.4.2 192.168.5.1
                        192.168.6.1
r3            (perm, OK) 23  IP  192.168.2.2 192.168.5.2
                        192.168.6.2 192.168.3.1
                        10.2.2.1
r4            (perm, OK) 23  IP  10.2.2.2
r5            (perm, OK) 23  IP  10.1.1.2

Hosts files on PCs are typically found in locations such as \Windows, \Etc, \Winnt\System32\Drivers\Etc and are named hosts. Reference Figure 3-22 for a sample Windows 2000 hosts file.

Figure 3-22 Windows 2000 Hosts File

The other files listed in Figure 3-22 can be quite helpful, too. Lmhosts is for NetBIOS name resolution. The other files contain common network, protocol, and port numbers. At the upper layers, I often ask myself whether a hostname or a NetBIOS issue exists. Figure 3-23 provides some examples, as does Table 3-1 earlier in this chapter.

Figure 3-23 Name and Address Resolution

DNS is hierarchical in nature; therefore, fully qualified domain names (FQDN) must be unique. Upper-level domain names include such domain names as com, edu, net, and gov. Second-level domain names must be registered. For example, ciscopress.com and shoretraining.com are both part of the top-level com domain. Cisco Press has authority over what they do under ciscopress.com, and I have control over what I do under shoretraining.com. Obviously, these second-level domains must be registered through Internet service providers (ISPs) or companies such as VeriSign.

The name space contains domains, subdomains, and hosts. The servers contain zones, which are database files with various record types. Common record types include alias (canonical name) or mail (MX) records.

Why do you care? Name resolution is certainly something to check for in the upper layers. If you can ping the IP address but not the hostname, for example, chances are you have a hostname resolution issue. If you can issue a Start > Run \\192.168.1.12 but not a Start > Run \\hostb, the problem is probably NetBIOS name resolution. These could in turn be DNS or WINS issues, depending on the applications and settings in your environment.

If you were capturing DNS information with Sniffer, you would see that queries typically are done over UDP port 53, whereas zone transfers are completed over TCP port 53. Next take a look at SNMP, which is everything but simple.

SNMP was originally for remote management of network hardware devices, but today is used for lots more. A management console sends a request to an agent (managed device) over UDP port 161, and the agent generates a trap on port 162 to a specific address. An agent really can't initiate anything on its own, but it can notify the manager of events, such as a link up or down or a software mail problem. The requests are part of a database referred to as a Management Information Base (MIB) in the SNMP world. Communities are groups that talk to one another to assist with security, and public is the default community. CiscoWorks, Cisco Info Center (CIC), and HP OpenView are examples of network management platforms that support SNMP-based management.

Next I review the RIP application. RIP is the routing protocol you set up earlier in the chapter so that one end system could get to another end system. RIP is an application that operates over UDP port 520. RIPv2 still operates over UDP port 520, but it uses 224.0.0.9 as a destination IP multicast address. Although each of the prefix entries includes a mask, RIPv2 is more considerate of the hosts, which are not interested in its messages, because their NICs will filter the frames at Layer 2. Figure 3-24 enables you to analyze the RIP header. Does my capture display RIPv1 or RIPv2?

Figure 3-24 RIP

You are correct if you said Figure 3-24 displays RIPv2 because the destination is the multicast 224.0.0.9 address rather than a local broadcast.

Now look at another application that is common in a day-to-day environment; Hypertext Transport Protocol (HTTP) uses TCP port 80 to provide web services. It also uses clear-text data transmission. Obviously this is a very big issue with purchasing items over the Internet. E-commerce applications make use of more secure protocols such as HTTPS over TCP port 443. If you compare a Sniffer trace of HTTP and HTTPS traffic, the HTTPS data is encrypted. Figure 3-25 shows you what people can sniff when you use HTTP to access a switch. I had to turn port monitoring on for this to work. Those details are covered more depth in the switch chapters, Chapter 6, "Shooting Trouble with CatOS and IOS," and Chapter 7, "Shooting Trouble with VLANs on Routers and Switches." For now, analyze the layered approach to HTTP as you have done with the other applications.

Figure 3-25 HTTP to a 1900 Switch

Now that I have touched on all the layers of the TCP/IP suite to lead into the addressing section, I will discuss DHCP. First there was RARP, then BOOTP, and now DHCP. The basic concept is the same. Take RARP, for example. It is used to resolve MAC addresses to IP addresses. It is the opposite of ARP, with which I know you have become pretty comfortable by now. The Bootstrap Protocol (BOOTP) was developed to allow diskless workstations to obtain IP information upon bootup. BOOTP spawned DHCP, which is widely used today.

DHCP is not fully automatic because someone must configure the server with a range of IP addresses (scope) and other optional parameters such as the mask, gateway, DNS server, WINS server, and so on. Clients request DHCP parameters via Layer 2 and Layer 3 broadcasts to UDP port 67. The server sends messages to the client on UDP port 68. However, it would defeat the purpose of a router if it were allowed to forward all broadcasts. On the other hand, it is possible for you to open up certain ports for routers to forward via the ip helper-address [ipaddress] command. Request forwarding is also possible via DHCP proxy agents. Figure 3-26 provides examples of the ip helper command. Helpers in effect change the local broadcast destination to a unicast or directed broadcast to reach the DHCP server. Table 3-8 shows DHCP frames that you should capture sometime with a protocol analyzer.

Figure 3-26 IP Helper

Table 3-8 DHCP Frames

DHCP Frame	Description
Discover	Client is looking for DHCP server (broadcast).
Offers	DHCP servers respond (broadcast).
Request	Client asks for first server that offered (broadcast).
Ack	DHCP server sends.
ARP	Client sends an ARP to check for duplicate addresses.

Addressing

Whether you use DHCP or static IP addressing, understanding addressing is a very important skill. As a CCNP Support candidate, assumed skills include basic things such as addressing, subnetting, summarizing, and routing. In this section I take the time to review because too many problems occur because of lack of planning with IP addressing. Proper planning prevents poor performance in all circumstances. Besides, you will need these skills in the Trouble Tickets, but more importantly in the real world.

Chapter 1 covered the basics of IP addressing, discussing the 4.2 billion (2³²), 32-bit, dotted-decimal format. Table 3-9 reviews classes and masks.

Table 3-9 IP Address Classes and Masks

	Class and Range	Decimal Mask	Bitwise Mask
0	A 1–1276	255.0.0.0	/8
10	B 128–191	255.255.0.0	/16
110	C 192–223	255.255.255.0	/24

Public addresses are registered through your ISP or at http://www.arin.net. ARIN is one of the three regional Internet registries (and is the authority in the United States). RIPE NCC is the authority for Europe, the Middle East, North Africa, and parts of Asia. APNIC is the Asia Pacific Network Information Centre (and is the authority for parts of Asia not under the authority of RIPE NCC). I registered my domain name information with VeriSign and my local ISP hosts the ShoreTraining.com website for me. Take a few minutes and go out to ARIN's website, use their whois tool to do a lookup of any public address you like. An example of a public address is 216.239.51.100, which happens to be one of my favorite search engines. This address falls within the network block of 216.239.32.0 to 216.239.63.255. However, I do not like to pick on public sites, so I will stay with private addresses for the practical studies. Private addresses should not be routed on the Internet, and they fall within the following ranges:

• 10.0.0.0/8 (10.0.0.0 to 10.255.255.255)

• 172.16.0.0/12 (172.16.0.0 to 172.31.255.255)

• 192.168.0.0/16 (192.168.0.0 to 192.168.255.255)

Notice that no matter whether you are using a public or private address, you still need a subnet mask to divide the network and host portion. I am certain you have heard many analogies on this by now, but the most common is probably the street being the network and your house number being a host on the network. Alternatively, look at the address and use your "network tape measure" to measure off the network bits from left to right, or just think of a painter masking off what he does not want to paint.

Look back at Figure 3-1 to review your streets (networks) and houses (hosts). Quickly make a list or table of your networks and hosts as in Table 3-10, because you will soon be changing your IP addressing scheme. Notice how I also included the router interfaces, because they need an address to operate with IP (unless, of course, you are using something like IP unnumbered).

Table 3-10 Current IP Addressing

Network	Address	Interface
192.168.1.0/24	192.168.1.1/24 192.168.1.11/24 192.168.1.12/24	r1e0 hosta >hostb
192.168.2.0/24	192.168.2.1/24 192.168.2.2/24	r1s1 r3s0/0
192.168.3.0/24	192.168.3.1/24 192.168.3.5/24	r3fa2/0 hostc
192.168.4.0/24	192.168.4.1/24 192.168.4.2/24	r1e1 r2e0
192.168.5.0/24	192.168.5.1/24 192.168.5.2/24	r2s1 r3s0/2
192.168.6.0/24	192.168.6.1/24 192.168.6.2/24	r2S0 r3s0/1
10.1.1.0/24	10.1.1.1/24 10.1.1.2/24	r1s0 r5s0
10.2.2.0/24	10.2.2.1/24 10.2.2.2/24	r3s0/3 r4s0/0

What a waste of address space, you should be thinking to yourself right about now, and if you were using public addresses, that would be more of a waste. Although the problem may not be apparent with a list of networks and hosts, many problems show up later. Keep your table handy; you will continue to examine this. First, however, I want to review subnetting in a little more detail.

Subnetting

From my back yard I can see the Chesapeake Bay Bridge and the eastbound and westbound cars. From Memorial Day to Labor Day is prime beach time. Every Friday night people head for the beach and return home on Sunday. Cars are almost at a standstill on the bridge at those times. Needless to say, this is one huge collision and broadcast domain. Sometimes the Mass Transit Authority (MTA) subnets, and I bet they don't even know it. They borrow temporarily less-congested lanes from the westbound side to send more people down the eastbound side. They also implemented EZPass to let the commuters have their own lanes (like queuing). That is all subnetting is. Now not as many cars (hosts) can travel the westbound side, because MTA borrowed them to make more lanes (subnets) for eastbound traffic. Subnetting is all about borrowing from the host bits to get more networks. Routers handle multiple subnets by partitioning collision and broadcast domains to avoid congestion.

Feel free to review the subnetting examples from Chapter 1, but you may as well put this to practice in your lab. Look at your routing tables to see how they display the networks you are using. Refer to Example 3-34 for mine.

Example 3-34 Chapter 3 Networks

r1#show ip route 
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
    i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default
    U - per-user static route, o - ODR
Gateway of last resort is not set
C  192.168.4.0/24 is directly connected, Ethernet1
R  192.168.5.0/24 [120/1] via 192.168.4.2, 00:00:23, Ethernet1
          [120/1] via 192.168.2.2, 00:00:18, Serial1
   10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
R    10.0.0.0/8 [120/2] via 192.168.4.2, 00:00:24, Ethernet1
R    10.2.2.0/24 [120/1] via 192.168.2.2, 00:00:18, Serial1
C    10.1.1.0/24 is directly connected, Serial0
R  192.168.6.0/24 [120/1] via 192.168.4.2, 00:00:24, Ethernet1
          [120/1] via 192.168.2.2, 00:00:18, Serial1
C  192.168.1.0/24 is directly connected, Ethernet0
C  192.168.2.0/24 is directly connected, Serial1
R  192.168.3.0/24 [120/1] via 192.168.2.2, 00:00:18, Serial1
r2>show ip route
...
C  192.168.4.0/24 is directly connected, Ethernet0
C  192.168.5.0/24 is directly connected, Serial1
   10.0.0.0/24 is subnetted, 2 subnets
R    10.1.1.0 [120/1] via 192.168.4.1, 00:00:08, Ethernet0
R    10.2.2.0 [120/1] via 192.168.6.2, 00:00:23, Serial0
         [120/1] via 192.168.5.2, 00:00:23, Serial1
C  192.168.6.0/24 is directly connected, Serial0
R  192.168.1.0/24 [120/1] via 192.168.4.1, 00:00:08, Ethernet0
R  192.168.2.0/24 [120/1] via 192.168.4.1, 00:00:08, Ethernet0
          [120/1] via 192.168.6.2, 00:00:23, Serial0
          [120/1] via 192.168.5.2, 00:00:23, Serial1
R  192.168.3.0/24 [120/1] via 192.168.6.2, 00:00:23, Serial0
          [120/1] via 192.168.5.2, 00:00:23, Serial1
r3>show ip route
...
R  192.168.4.0/24 [120/1] via 192.168.2.1, 00:00:17, Serial0/0
          [120/1] via 192.168.6.1, 00:00:11, Serial0/1
          [120/1] via 192.168.5.1, 00:00:11, Serial0/2
C  192.168.5.0/24 is directly connected, Serial0/2
   10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
R    10.1.1.0/24 [120/1] via 192.168.2.1, 00:00:17, Serial0/0
R    10.0.0.0/8 [120/2] via 192.168.6.1, 00:00:11, Serial0/1
          [120/2] via 192.168.5.1, 00:00:11, Serial0/2
C    10.2.2.0/24 is directly connected, Serial0/3
C  192.168.6.0/24 is directly connected, Serial0/1
R  192.168.1.0/24 [120/1] via 192.168.2.1, 00:00:18, Serial0/0
C  192.168.2.0/24 is directly connected, Serial0/0
C  192.168.3.0/24 is directly connected, FastEthernet2/0
r4>show ip route
...
R  192.168.4.0/24 [120/2] via 10.2.2.1, 00:00:13, Serial0/0
R  192.168.5.0/24 [120/1] via 10.2.2.1, 00:00:14, Serial0/0
   10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
R    10.1.1.0/24 [120/2] via 10.2.2.1, 00:00:14, Serial0/0
R    10.0.0.0/8 [120/3] via 10.2.2.1, 00:00:14, Serial0/0
C    10.2.2.0/24 is directly connected, Serial0/0
R  192.168.6.0/24 [120/1] via 10.2.2.1, 00:00:14, Serial0/0
R  192.168.1.0/24 [120/2] via 10.2.2.1, 00:00:14, Serial0/0
R  192.168.2.0/24 [120/1] via 10.2.2.1, 00:00:14, Serial0/0
R  192.168.3.0/24 [120/1] via 10.2.2.1, 00:00:14, Serial0/0
r5>show ip route
...
R  192.168.4.0/24 [120/1] via 10.1.1.1, 00:00:05, Serial0
R  192.168.5.0/24 [120/2] via 10.1.1.1, 00:00:06, Serial0
   10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
R    10.2.2.0/24 [120/2] via 10.1.1.1, 00:00:06, Serial0
R    10.0.0.0/8 [120/3] via 10.1.1.1, 00:00:06, Serial0
C    10.1.1.0/24 is directly connected, Serial0
R  192.168.6.0/24 [120/2] via 10.1.1.1, 00:00:06, Serial0
R  192.168.1.0/24 [120/1] via 10.1.1.1, 00:00:06, Serial0
R  192.168.2.0/24 [120/1] via 10.1.1.1, 00:00:06, Serial0
R  192.168.3.0/24 [120/2] via 10.1.1.1, 00:00:06, Serial0
r5>

You can apply the shaded output in the preceding example to the other routers, but on r1 I am illustrating that there are multiple paths to subnet 192.168.5.0 and 192.168.6.0. Note how the routing table output shows 10.0.0.0/8, which is the classful mask, with the subnets 10.1.1.0/24 and 10.2.2.0/24 beneath it.

To prepare for the chapter Trouble Tickets, I want you to plan your new addressing scheme using 192.168.5.0/24 as a starting point. You know how many networks and hosts you have, but plan on adding a few more for growth. Do not assume that you can use subnet 0 (the first subnet), the last subnet, or VLSM for now. Write out your calculations as I do in Figure 3-27, but save the actual configuration for the chapter Trouble Tickets. Feel free to expand the binary to truly help you master the subnetting concepts.

Figure 3-27 Subnetting

You certainly should plan your addresses carefully, and this is more than just compiling an Excel spreadsheet and checking off what you have handed out to everyone. However, what could you do if I told you that I want to take one of the subnets and subdivide it to maximize the number of subnets I can possibly get from it? I tend to reserve subnet 0 for this purpose, so take 192.168.5.0/28 and further subnet it according to the information provided. Figure 3-28 shows my VLSM calculations.

Figure 3-28 VLSM

I took the VLSM example from a different approach in that I knew I never needed more than two hosts. So I used the 2^x – 2 formula to calculate hosts and placed the 0s in the chart first. Tables are helpful to organize such data, as I demonstrate in Table 3-11.

Table 3-11 Subnets, Broadcast Addresses, Hosts, and VLSM Subnets

Subnets	0	16	32	48	64	80	96	...	224
Broadcast	15	31	47	63	79	95	111	...	239
Hosts	1– 14	17–33	33–46	49–62	65–78	81–94	97–110	...	225–238
VLSM subnets (hosts)	0 (1–2) 4 (5–6) 8 (9–10) 12 (13–14)

Some general suggestions I can recommend for VLSM include the following:

Step 1	Start with your host requirements.
Step 2	Next calculate for the LAN segments.
Step 3	Then subnet a LAN to get your WANs.
Step 4	More than three levels can get a little too confusing.

Ultimately, besides being neat about things and saving addresses, you want to plan your address scheme so that it is not so stressful on your routing tables. It would be a perfect world if everyone really understood the importance of this. After all, there is more to life than just sharing your cake; you must eat it too. Figure 3-29 offers a hierarchical view of the subnetting and VLSM math you did previously, which will in turn make summarizing a breeze.

Figure 3-29 Summarization

Summarization

Summarization is the real reason you want to pay attention to your addressing scheme. Besides only having a finite number of addresses, routing table growth has just exploded over the years. There are more than 100,000 Internet routes to date. See for yourself in Figures 3-30 and 3-31 or go to a similar site for a more current report. Summarizing enables you to have a single IP address range represent a collection of smaller ranges when deployed hierarchically. Collapsing the routing table is an obvious advantage to summarization, but think about the impact on my network if my routing tables are affected every time a link goes down on your network.

Summarization limits the number of routers that need to recalculate routing tables. To determine a summary route, a router determines the high-order (1) bits that match for all addresses. Sometimes this is done on the classful boundary automatically and other times you and I may need to include the appropriate summary statement, depending on the routing protocol.

Figure 3-30 Route Server

Figure 3-31 Classless Interdomain Routing (CIDR) Report

Subnetting, VLSM, summarization, aggregation, supernetting, and CIDR are all about matching bits. Subnetting and VLSM move the network bit boundary to the right, whereas summarization, aggregation, supernetting, and CIDR move the network bit boundary to the left. In Figure 3-32, HQ starts with 192.168.5.0/24 and breaks it down into subnet 192.168.5.32/27 for the Northern Region, 192.168.5.64/27 for the Western Region, 192.168.5.96/27 for the Southern Region, and 192.168.5.128/27 for the Eastern Region. In turn, each of the regions uses VLSM to subnet their Ethernets to a /28 and their serials to a /30. Yet each region can summarize the /27 back to HQ Core.

Figure 3-32 Network Bit Boundaries

The practical addressing examples certainly bring out the mathematics involved, but decreasing the impact on networks attached to the routers and ultimately the hosts connected to various switches is the goal. I address that topic a bit more in Part III, "Supporting Ethernet, Switches, and VLANs." Now I want to quickly review routing protocols so that you can shoot their troubles throughout the book.

Routing Protocols

Routing protocols have their own unique characteristics, and various Layer 2 encapsulation types have a big impact on them. Understanding how they work will certainly assist you in troubleshooting them now and later. Most of the statistics listed for each can be found with IOS commands such as show ip protocols, show ip route, and show ip routing-protocol ?. Logging, debug, and protocol analyzers all with various levels of impact on the working environment are certainly valuable tools to see more detail when troubleshooting, too. However, I want you to reserve them for later chapters. After all, many of these protocols are not just IP-specific.

First I discuss the following Interior Gateway Protocols (IGPs): RIP, Interior Gateway Routing Protocol (IGRP)/Enhanced Interior Gateway Routing Protocol (EIGRP), OSPF, and IS-IS. Then I review Border Gateway Protocol (BGP), which is an Exterior Gateway Protocol (EGP).

RIP

Routing Information Protocol (RIP) was originally designed for Xerox PARC Universal Protocol (PUP), and in many ways is still for "pups." It was called GWINFO in the Xerox Network Systems (XNS) protocol suite in 1981, and defined in RFC 1058 in 1988. It is easy to configure, and it works very well in small networks. In larger networks, however, it can be less effective; as I say to myself, "It can RIP you apart." There are alternatives to RIP for larger environments.

Everyone knows RIP because it has been widely adopted by PC, UNIX, and router makers alike. RIP has disadvantages in that it operates over UDP port 520 and the maximum hop count is 15. RIPv2 assists with the broadcast nature in that it operates via multicast over 224.0.0.9. Both RIPv1 and RIPv2 are distance vector routing protocols, which are often referred to as routing by rumor protocols.

Examine the following list of RIP characteristics and refer back to the examples throughout the chapter, for you have already experimented with RIPv1 and RIPv2. I hope your practical exercises, with RIPv1 not supporting discontiguous subnets and RIPv2 supporting them, will stay with you for a long time.

The following are RIP characteristics:

• Open protocol, widely used, stable.

• Good for small networks in that it is very easy to configure.

• There are RIP-like distance vector routing protocols for Novell and AppleTalk.

• Distance vector routing protocol.

• IGP.

• IP RIP updates are sent every 30 seconds via broadcast (224.0.0.9 for RIPv2).

• UDP port 520

• Administrative distance is 120.

• Single metric is hop count. (The limit is 15 to assist with count-to-infinity.)

• Timers help regulate performance:

  • Update timer—Frequency of routing updates. Every 30 seconds IP RIP sends a complete copy of its routing table, subject to split horizon. (IPX RIP does this every 60 seconds.)

  • Invalid timer—Absence of refreshed content in a routing update. RIP waits 180 seconds to mark a route as invalid and immediately puts it into holddown.

  • Hold-down timers and triggered updates—Assist with stability of routes in the Cisco environment. Holddowns ensure that regular update messages do not inappropriately cause a routing loop. The router doesn't act on nonsuperior new information for a certain period of time. RIP's hold-down time is 180 seconds.

  • Flush timer—RIP waits an additional 240 seconds after holddown before it actually removes the route from the table.

• Other stability features to assist with routing loops include the following:

  • Split horizon—Not useful to send information about a route back in the direction from which it came.

  • Poison reverse—Updates that are sent to invalidate a route and place it in holddown.

• Bellman-Ford algorithm.

• RIPv2 supports VLSM and summarization. (RIPv1 doesn't.) RIPv2 always autosummarizes at the class boundary.

RIP maintains only the best route in its routing table, as you can verify in examples throughout the chapter. In Example 3-2, for example, I configured RIPv1 on r1 and continued to configure the other routers. In Example 3-15, while troubleshooting I looked at the routing tables and IP routing protocols. Then I configured RIPv2 in Example 3-21. Example 3-22 displays the routing tables and output of show ip protocols with RIPv2 configured. Look back at these examples to review such things as the update characteristics, timers, and administrative distance associated with RIP.

Example 3-35 illustrates routes in holddown. First I turn on service time stamps and set the clock so that you can see the actual timing of events. Then I turn on debug ip rip events, remove the cable from router 2 Serial 1, plug it back in, watch the routing updates, and view the appropriate routing tables.

Example 3-35 Debug Output for RIPv2 Packets

r1(config)#service timestamps debug datetime localtime
r1(config)#end
r1#clock set 3:22:00 11 October 2002
r1#copy running-config startup-config
Destination filename [startup-config]? 
Building configuration...
[OK]
r1#debug ip rip events
RIP event debugging is on
Oct 11 03:22:52: RIP: sending v2 update to 224.0.0.9 via Ethernet0 (192.168.1.1)
Oct 11 03:22:52: RIP: Update contains 8 routes
Oct 11 03:22:52: RIP: Update queued
Oct 11 03:22:52: RIP: sending v2 update to 224.0.0.9 via Ethernet1 (192.168.4.1)
Oct 11 03:22:52: RIP: Update sent via Ethernet0
Oct 11 03:22:52: RIP: Update contains 5 routes
Oct 11 03:22:52: RIP: Update queued
Oct 11 03:22:52: RIP: sending v2 update to 224.0.0.9 via Serial0 (10.1.1.1)
Oct 11 03:22:52: RIP: Update sent via Ethernet1
Oct 11 03:22:52: RIP: Update contains 8 routes!
Oct 11 03:22:52: RIP: Update queued
Oct 11 03:22:52: RIP: sending v2 update to 224.0.0.9 via Serial1 (192.168.2.1)
Oct 11 03:22:52: RIP: Update sent via Serial0
Oct 11 03:22:52: RIP: Update contains 4 routes
Oct 11 03:22:52: RIP: Update queued
Oct 11 03:22:52: RIP: Update sent via Serial1
Oct 11 03:22:59: RIP: received v2 update from 192.168.2.2 on Serial1
Oct 11 03:22:59: RIP: Update contains 5 routes
Oct 11 03:23:02: RIP: received v2 update from 192.168.4.2 on Ethernet1
Oct 11 03:23:02: RIP: Update contains 4 routes
r1#!!!now I will unplug the r2s1 cable
r1#
Oct 11 03:23:14: RIP: received v2 update from 192.168.2.2 on Serial1
Oct 11 03:23:14: RIP: Update contains 5 routes
Oct 11 03:23:14: RIP: received v2 update from 192.168.4.2 on Ethernet1
Oct 11 03:23:14: RIP: Update contains 4 routes
Oct 11 03:23:14: RIP: sending v2 update to 224.0.0.9 via Ethernet0 (192.168.1.1)
Oct 11 03:23:14: RIP: Update contains 8 routes
Oct 11 03:23:14: RIP: Update queued
...
r1#show ip route
...
C  192.168.4.0/24 is directly connected, Ethernet1
R  192.168.5.0/24 is possibly down, routing via 192.168.4.2, Ethernet1
   10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
R    10.2.2.0/24 [120/1] via 192.168.2.2, 00:00:10, Serial1
R    10.0.0.0/8 [120/2] via 192.168.4.2, 00:00:07, Ethernet1
C    10.1.1.0/24 is directly connected, Serial0
R  192.168.6.0/24 [120/1] via 192.168.4.2, 00:00:07, Ethernet1
          [120/1] via 192.168.2.2, 00:00:11, Serial1
C  192.168.1.0/24 is directly connected, Ethernet0
C  192.168.2.0/24 is directly connected, Serial1
R  192.168.3.0/24 [120/1] via 192.168.2.2, 00:00:11, Serial1

Plug the cable back in, continue to review the results, and turn off all debug activity, as in Example 3-36.

Example 3-36 Plug the Cable Back In and Observe the Results

r1#!!!now I will plug the cable back in
...
r1#show ip route
C  192.168.4.0/24 is directly connected, Ethernet1
R  192.168.5.0/24 is possibly down, routing via 192.168.4.2, Ethernet1
   10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
R    10.2.2.0/24 [120/1] via 192.168.2.2, 00:00:13, Serial1
...
Oct 11 03:26:16: RIP: received v2 update from 192.168.4.2 on Ethernet1
Oct 11 03:26:16: RIP: Update contains 4 routes
Oct 11 03:26:16: RIP: sending v2 update to 224.0.0.9 via Ethernet0 (192.168.1.1)
Oct 11 03:26:16: RIP: Update contains 8 routes
Oct 11 03:26:16: RIP: Update queued
Oct 11 03:26:16: RIP: sending v2 update to 224.0.0.9 via Ethernet1 (192.168.4.1)
Oct 11 03:26:16: RIP: Update sent via Ethernet0
Oct 11 03:26:16: RIP: Update contains 5 routes
Oct 11 03:26:16: RIP: Update queued
...
r1#show ip route
...
C  192.168.4.0/24 is directly connected, Ethernet1
R  192.168.5.0/24 [120/1] via 192.168.4.2, 00:00:04, Ethernet1
   10.0.0.0/8 is variably subnetted, 3 subnets, 2 masks
R    10.2.2.0/24 [120/1] via 192.168.2.2, 00:00:07, Serial1
R    10.0.0.0/8 [120/2] via 192.168.4.2, 00:00:04, Ethernet1
C    10.1.1.0/24 is directly connected, Serial0
R  192.168.6.0/24 [120/1] via 192.168.4.2, 00:00:04, Ethernet1
          [120/1] via 192.168.2.2, 00:00:07, Serial1
C  192.168.1.0/24 is directly connected, Ethernet0
C  192.168.2.0/24 is directly connected, Serial1
R  192.168.3.0/24 [120/1] via 192.168.2.2, 00:00:07, Serial1
r1#undebug all

Fix any issues before you continue. If you cannot see the debug output when you telnet in, you may need to turn on terminal monitor (term mon). In this example, I did not show the output of the interface status on r2, but that is pretty important in troubleshooting such issues. Repeat the example as necessary or log it for future reference.

See Figure 3-24 or look at one of your Sniffer traces to analyze the RIP packet format. Up to 25 destinations can be listed in a single packet. Next I discuss the Cisco proprietary routing protocols IGRP and EIGRP.

IGRP/EIGRP

Interior Gateway Routing Protocol (IGRP) was developed in the mid-1980s as a Cisco proprietary protocol to help overcome some of limitations of RIP, such as the single metric of hop count. It has stability features similar to RIP—hold-down timers, split horizon, poison reverse, and triggered updates. The timers are as follows: invalid 270 seconds, holddown 280 seconds, and flush 630 seconds. It also contains mechanisms to influence route selection and unequal load sharing. I use the phrase Big Dogs Really Like Meat to remember the metrics for IGRP and EIGRP:

• Bandwidth

• Delay

• Reliability

• Load

• MTU

IGRP is an IGP, a distance vector routing protocol based on the Bellman-Ford algorithm that broadcasts routing updates every 90 seconds over IP protocol number 9. It is fine for small and medium-size networks, but Cisco enhanced it greatly and added VLSM support to its replacement, EIGRP.

Cisco developed Enhanced IGRP (EIGRP) in the early 1990s to overcome limitations of RIP and its own IGRP. Cisco says IGRP is going to be removed from IOS. EIGRP is suitable for large networks today and supports multiple routed protocols. It consumes significantly less bandwidth because of its partial, bounded updates, and can be one of the fastest converging routing protocols there is.

The following are EIGRP characteristics:

• Cisco proprietary protocol.

• Good for small to large networks.

• Very easy to configure. Uses autonomous system (AS) number.

• Supports multiple Layer 3 routed protocol stacks, such as IP, Novell IPX, and AppleTalk.

• Advanced distance vector routing protocol. Often called a hybrid due to its incremental updates and rapid convergence capabilities.

• IGP.

• Multicast triggered updates over 224.0.0.10, not periodic.

• IP protocol number 88.

• Internal administrative distance is 90; external is 170.

• Metrics are bandwidth, delay, reliability, load, and MTU.

• Supports equal- and unequal-cost load sharing.

• Other stability features to assist with routing loops:

  • Split horizon—Not useful to send information about a route back in the direction from which it came

  • Poison reverse—Updates that are sent to remove a route and place it in holddown

• Uses Diffusing Update algorithm (DUAL) to select loop-free paths and give it very fast convergence.

• Supports VLSM and manual summarization (classless).

• Automatic classful boundary summarization.

• Manual summarization on update sent out each interface.

• Automatic redistribution with IGRP if same AS number.

• Route tagging for policy-based routing.

EIGRP gets its reliability from the Reliable Transport Protocol (RTP). It maintains not only a routing table, but also a neighbor and topology table. EIGRP maintains alternate routes referred to as successors (routing table) and feasible successors (topology table) to quickly converge. The following packet types are used for neighbor communications: hellos (multicast) and acks (unicast), update (multicast or unicast), query (multicast), reply (unicast), and request (multicast or unicast). Packets are held in a queue for retransmission, and there are separate neighbor tables (and entirely separate processes) for each protocol.

Cisco has some wonderful white papers on EIGRP that you should download and review to assist you with troubleshooting. Other popular references include the book EIGRP Network Design Solutions (Cisco Press). Just remember when troubleshooting EIGRP, active ain't good, but passive is. Active means that you are actively looking for something that you don't have. Pay particular attention to summarization when you are experiencing stuck-in-active situations. Besides the basic IOS commands in your repertoire, you should add show ip eigrp ? and debug ip eigrp ? for your EIGRP IP troubleshooting assistance. Next I very briefly review OSPF, IS-IS, and BGP. You will continue to configure, analyze, and troubleshoot the convergence of these protocols over various Layer 2 technologies throughout the rest of this book.

OSPF

Open Shortest Path First (OSPF) overcomes the disadvantages of RIP and is not proprietary in nature, but its openness supports only the IP routed protocol. The protocol is a link-state IGP based on the Dijkstra algorithm developed by the Internet Engineering Task Force (IETF) to support large heterogeneous networks. Lots of research was completed from 1987 until the current OSPFv2 specification in 1991. Link-state advertisements are sent to all, which causes an initial flood on the router; after that, however, OSPF is very efficient in operation. It uses three different databases (tables) for the neighbors, link states, and routes.

The following are OSPF characteristics:

• Open protocol.

• Good for small to large networks.

• Not as easy to design and configure as other protocols.

• Supports only the IP Layer 3 routed protocol stack.

• Link-state routing protocol (doesn't just send to neighbors like distance vector).

• IGP.

• Multicast link-state advertisement (LSA) updates over 224.0.0.5 and 224.0.0.6.

• IP protocol number 89.

• Administrative distance is 110.

• Metric is a cumulative cost (inversely proportional to bandwidth).

• Supports only equal-cost load sharing, but some implementations can take advantage of type of service (TOS) requests.

• Requires a routing hierarchy in that every area must touch the backbone area (otherwise temporary fixes such as virtual links are used). Various router types, LSA types, area types, and states, depending on your design and Layer 2 topology.

• Uses Dijkstra algorithm to select loop-free paths and give it fast convergence. This uses LSAs and is based on the Shortest Path First (SPF) algorithm, where the protocol got its name.

• Supports VLSM and summarization (classless).

• Supports manual summarization only; this is not automatic like EIGRP. Must be performed on an ABR (area range) or ASBR (summary address) only.

• Route tagging for policy-based routing.

Besides the basic IOS commands in your repertoire, you should add show ip ospf ? and debug ip ospf ? for your OSPF troubleshooting assistance.

OSPF references include the books OSPF Network Design Solutions (Cisco Press) and OSPF: Anatomy of an Internet Routing Protocol (Addison-Wesley). John Moy is the author of the latter, and if you truly want the RFC detail, this is the book to read. Next take a look at IS-IS.

IS-IS

The ISO was working on Intermediate System-to-Intermediate System (IS-IS) about the same time the IAB was working on OSPF. The late 1980s and early 1990s, a time in our history when everyone thought the OSI suite would overtake TCP/IP, was when Integrated IS-IS was proposed. Although originally designed for OSI routing, IS-IS was developed by ISO to support CLNS/CLNP. Integrated IS-IS, which supports IP, was a later development. The purpose was to provide a single routing protocol that could route Connectionless Network Protocol (CLNS) and IP. IS-IS is in use by ISPs today. OSPF and IS-IS have many common features.

The following are IS-IS characteristics:

• Open protocol.

• Good for medium to very large networks.

• ISO link-state routing protocol similar to OSPF.

• IGP.

• IS-IS Layer 2 PDUs rather than IP packets.

• Uses Layer 2 multicast.

• Administrative distance is 115.

• Very limited metric dynamic range (0–63).

• Equal-cost load sharing.

• Two-level hierarchical topology.

• Uses Dijkstra/SPF algorithm.

• Supports VLSM and summarization.

• Manual summarization.

• Route tagging for policy-based routing.

Besides the basic IOS commands in your repertoire, you should add show is-is ?, show clns ?, and debug is-is ? for your IS-IS troubleshooting assistance.

Jeff Doyle's Routing TCP/IP, Volume I does a good job of explaining many topics, including IS-IS. Volume II is great for BGP, as is Sam Halabi's Internet Routing Architectures.

BGP

Border Gateway Protocol (BGP) is an EGP that pretty much replaces the legacy EGP protocol itself. BGP performs routing between autonomous systems and is the standard routing protocol on the Internet. This is referred to as External BGP (EBGP), whereas when BGP is used to route within an AS it is referred to as Interior (IBGP). BGP is not a routing protocol for the fainthearted. It requires all manual configurations for a very good reason; you are not only affecting you, but you are also affecting me. For troubleshooting BGP, if routes are not in the BGP table, there is no way they will be in the routing table. Always make sure your neighbors are talking to you. One of the most useful commands in troubleshooting BGP is show ip bgp summary.

The following are BGP characteristics:

• Open protocol.

• Good for very large internetworks.

• Not as easy to design and configure as other protocols. Everything is manual, including neighbors (peers).

• Advanced distance vector or path vector routing protocol.

• EGP.

• TCP port 179.

• Internal administrative distance is 200; external is 20.

• Metrics include many attributes such as MED, Origin, AS-Path, Next-hop, and Community.

• Does not demand a particular routing hierarchy; roll your own.

• Automatic and manual summarization features.

• Route tagging for policy-based routing.

Besides the basic IOS commands in your repertoire, you should add show ip bgp ? and debug ip bgp ? for your BGP troubleshooting assistance. Remember that BGP is an EGP and that what you do affects others. ISPs often turn on dampening to account for those who really don't know what they are doing with BGP configuration or to compensate for link flaps. Too many changes within a certain period of time may mean that you don't get to communicate at all.

Individual routing protocols are books in themselves. However, I have quickly summarized the common routing protocols for you. The routing table is a good place to start troubleshooting, but if routes are missing, ultimately they may depend on Physical Layer or Data Link Layer issues, neighbor relationships, and topology or link-state tables. Cisco does a great job at categorizing more specific IP routing issues for you (see Figure 3-33).

Figure 3-33 IP Routing Top Issues

Now it is time for the Trouble Tickets. The plan here is to give you several things to do, let you make mistakes and fix some things on your own, and to introduce other problems that you should have some experience with as a support person. Shooting trouble with IP can easily be a multiple-volume set of books in itself. Because it is a big part of troubleshooting today, I also integrate more IP-related issues into the other chapters.

Trouble Tickets

Complete the following Trouble Tickets in order. Use the tools from this and the preceding chapters to analyze, test, and document as you go. Feel free to create your own Physical Layer problems if you need more practice there. Sample solutions are provided after this section.

Trouble Ticket 1

Quickly review your existing configurations and copy them to a TFTP server as r1-rip2, r2-rip2, r3-rip2, r4-rip2, and r5-rip2 so that you can use them again later.

Trouble Ticket 2

Draw a new Chapter 3 scenario diagram and label the address ranges and interfaces per your calculations in Figure 3-27. Leave the links to r4 and r5 numbered as they are.

Trouble Ticket 3

Re-address the network using your new diagram. Remove RIP and save your configurations.

Trouble Ticket 4

Change the routing protocol from RIP to EIGRP using an AS number of choice. Verify routing processes, protocols, and that you have the correct information in each routing table. Disconnect and reconnect the r1s1 cable and experiment a bit.

Trouble Ticket 5

Ping and trace from hosta to hostc and capture the results. Save all configurations and verify everything before you continue. Quickly review your configurations and copy them to a TFTP server as r1-eigrp, r2-eigrp, r3-eigrp, r4-eigrp, and r5-eigrp so that you can use them again later.

Trouble Ticket 6 (Optional)

Set up an access list to allow only hosta to telnet to r1. The other hosts should not be able to telnet, but should be able to ping or trace. Apply and test the ACL. Leave the ACL configuration, but remove the application of it from the interface after you verify that it works.

Trouble Ticket 7 (Optional)

Set up one of your clients for DHCP to capture and analyze the Discover, Offers, Request, and Acknowledgement frames.

Trouble Tickets Solutions

These solutions are not always the only way to perform these tasks. However, the upcoming chapter scenarios are based on these solutions.

Trouble Ticket 1 Solution

I reviewed and copied my existing configurations to my TFTP server as rx-rip2, where I replaced x for each router number. In Example 3-37, I performed a simple ping first to make sure I had connectivity to the TFTP server, and then copied the files. You may want to tweak the configuration of your TFTP server to place the files in the location you prefer for easy access later. You should also get familiar with how your particular TFTP server handles files that already exist. For example, you may want to have it prompt you as to what to do if the files already exist. The answer shows the output for r1 only, but this was performed from each router.

Example 3-37 Copying Existing Configurations to a TFTP Server

r1#ping 192.168.1.11
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.11, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 1/3/4 ms
r1#copy running-config tftp
Address or name of remote host []? 192.168.1.11
Destination filename [running-config]? r1-rip2
!!
1169 bytes copied in 5.680 secs (233 bytes/sec)
r1#

Trouble Ticket 2 Solution

Figure 3-34 displays the new addressing scheme that I used as a guide for configurations in the examples to follow.

Figure 3-34 Trouble Tickets Scenario

Trouble Ticket 3 Solution

I used Figure 3-34 as a guide to re-address the network. Example 3-38 shows a few problems I had when re-addressing the interfaces. I could have typed no ip address on each interface to remove the old one before I started, but then you may not have realized the overlapping problems. While on each router, I also issued no router rip to quickly remove the RIP routing protocol commands. A quick test after the configuration is to perform show ip interface brief to verify the configuration against the drawing.

Example 3-38 Re-Addressing and Removing RIP

r1(config)#interface e0
r1(config-if)#ip address 192.168.5.17 255.255.255.240
r1(config-if)#interface e1
r1(config-if)#ip address 192.168.5.33 255.255.255.240
r1(config-if)#interface s1
r1(config-if)#ip add 192.168.5.81 255.255.255.240
r1(config-if)#no router rip
r1(config)#end
r1#copy running-config startup-config
r2(config)#interface e0
r2(config-if)#ip address 192.168.5.34 255.255.255.240
192.168.5.32 overlaps with Serial1
r2(config-if)#interface s1
r2(config-if)#ip address 192.168.5.49 255.255.255.240
r2(config-if)#interface e0
r2(config-if)#ip address 192.168.5.34 255.255.255.240
r2(config-if)#interface s0
r2(config-if)#ip address 192.168.5.65 255.255.255.240
r2(config-if)#no router rip
r2(config)#end
r2#copy running-config startup-config
r3(config)#interface s0/0
r3(config-if)#ip address 192.168.5.82 255.255.255.240
192.168.5.80 overlaps with Serial0/2
r3(config-if)#interface s0/2
r3(config-if)#ip address 192.168.5.50 255.255.255.0
r3(config-if)#interface s0/0
r3(config-if)#ip address 192.168.5.82 255.255.255.240
192.168.5.80 overlaps with Serial0/2
r3(config-if)#interface s0/2
r3(config-if)#ip address 192.168.5.50 255.255.255.240
r3(config-if)#interface s0/0
r3(config-if)#ip address 192.168.5.82 255.255.255.240
r3(config-if)#interface s0/1
r3(config-if)#ip address 192.168.5.66 255.255.255.240
r3(config-if)#interface fa2/0
r3(config-if)#ip address 192.168.5.97 255.255.255.240
r3(config-if)#no router rip
r3(config)#end
r3#copy running-config startup-config
r4(config)#no router rip
r4(config)#end
r4#copy running-config startup-config
r5(config)#no router rip
r5(config)#end
r5#copy running-config startup-config
r1#show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Ethernet0       192.168.5.17  YES manual up          up   
Ethernet1       192.168.5.33  YES manual up          up   
Serial0         10.1.1.1      YES NVRAM  up          up   
Serial1         192.168.5.81  YES manual up          up   
r2#show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Ethernet0       192.168.5.34  YES manual up          up   
Serial0         192.168.5.65  YES manual up          up   
Serial1         192.168.5.49  YES manual up          up   
r3#show ip interface brief
Interface       IP-Address   OK? Method Status        Protocol
Serial0/0       192.168.5.82  YES manual up          up   
Serial0/1       192.168.5.66  YES manual up          up   
Serial0/2       192.168.5.50  YES manual up          up   
Serial0/3       10.2.2.1      YES NVRAM  up          up   
...
FastEthernet2/0    192.168.5.97  YES manual up          up

Trouble Ticket 4 Solution

Example 3-39 starts the EIGRP configuration and testing. I used AS number 500 for my test and saved my configurations as I went along. Notice in Example 3-39 how I configured 10.1.1.0 and 10.2.2.0. show running-config shows 10.0.0.0. EIGRP is relatively easy to configure, but remember that it is proprietary.

Example 3-39 EIGRP Configuration

r1(config)#router eigrp 500
r1(config-router)#network 192.168.5.0
r1(config-router)#network 10.1.1.0
r1(config-router)#end
r1#copy running-config startup-config
r2(config)#router eigrp 500
r2(config-router)#network 192.168.5.0
r2(config-router)#end
r2#copy running-config startup-config
r3(config)#router eigrp 500
r3(config-router)#network 192.168.5.0
r3(config-router)#network 10.2.2.0
r3(config-router)#end
r3#copy running-config startup-config
r4(config)#router eigrp 500
r4(config-router)#network 10.2.2.0
r4(config-router)#end
r4#copy running-config startup-config
r5(config)#router eigrp 500
r5(config-router)#network 10.1.1.0
r5(config-router)#end
r5#copy running-config startup-config
r5#show running-config
...
router eigrp 500
 network 10.0.0.0
!
ip classless
!
...
end

Example 3-40 illustrates show ip protocols and show ip route to verify EIGRP parameters and routes. You should verify every router; I have listed only the first one. The EIGRP learned routes are shown with a D, whereas RIP represented them with an R. It is important to note how EIGRP automatically summarizes on the classful boundary.

Example 3-40 EIGRP Testing with show ip protocols and show ip route

r1>show ip protocols
Routing Protocol is "eigrp 500"
 Outgoing update filter list for all interfaces is not set
 Incoming update filter list for all interfaces is not set
 Default networks flagged in outgoing updates
 Default networks accepted from incoming updates
 EIGRP metric weight K1=1, K2=0, K3=1, K4=0, K5=0
 EIGRP maximum hopcount 100
 EIGRP maximum metric variance 1
 Redistributing: eigrp 500
 Automatic network summarization is in effect
 Automatic address summarization:
  10.0.0.0/8 for Ethernet0, Ethernet1, Serial1
   Summarizing with metric 40512000
  192.168.5.0/24 for Serial0
   Summarizing with metric 281600
 Routing for Networks:
  10.0.0.0
  192.168.5.0
 Routing Information Sources:
  Gateway     Distance   Last Update
  (this router)     5   00:16:31
  10.1.1.2       90   00:15:44
  192.168.5.82     90   00:16:31
  Gateway     Distance   Last Update
  192.168.5.34     90   00:16:32
 Distance: internal 90 external 170
r1>show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
    i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default
    U - per-user static route, o - ODR
Gateway of last resort is not set
   192.168.5.0/24 is variably subnetted, 7 subnets, 2 masks
D    192.168.5.96/28 [90/40514560] via 192.168.5.82, 00:16:37, Serial1
D    192.168.5.64/28 [90/40537600] via 192.168.5.34, 00:16:37, Ethernet1
C    192.168.5.80/28 is directly connected, Serial1
C    192.168.5.32/28 is directly connected, Ethernet1
D    192.168.5.48/28 [90/40537600] via 192.168.5.34, 00:16:37, Ethernet1
D    192.168.5.0/24 is a summary, 00:18:11, Null0
C    192.168.5.16/28 is directly connected, Ethernet0
   10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
D    10.0.0.0/8 is a summary, 00:16:37, Null0
C    10.1.1.0/24 is directly connected, Serial0

Example 3-41 further explores EIGRP with some EIGRP-specific IOS debug commands. I disconnected the r1S1 cable after turning on a pretty explicit command for you to see the EIGRP packet and update process. After looking at the routing table, I plugged the cable back in and explored some of the EIGRP show and logging commands that are quite helpful in troubleshooting in Example 3-42.

Example 3-41 Experimenting with EIGRP Troubleshooting Tools

r1#debug eigrp ?
 fsm    EIGRP Dual Finite State Machine events/actions
 neighbors EIGRP neighbors
 packets  EIGRP packets
 transmit  EIGRP transmission events
r1#debug eigrp transmit
EIGRP Transmission Events debugging is on
  (ACK, PACKETIZE, STARTUP, PEERDOWN, LINK, BUILD, STRANGE, DETAIL)
01:21:06: %LINK-3-UPDOWN: Interface Serial1, changed state to down
01:21:06: DNDB QUERY 192.168.5.80/28, serno 3 to 16, refcount 3
01:21:06:  Anchoring Serial0, starting Serial0 timer
01:21:06:  Anchoring Ethernet1, starting Ethernet1 timer
01:21:06:  Anchoring Serial1, starting Serial1 timer
01:21:06: Peer 192.168.5.82 going down
01:21:06: DNDB QUERY 192.168.5.96/28, serno 15 to 17, refcount 3
01:21:06: Last peer deleted from Serial1
01:21:06:  Dropping refcount on 192.168.5.80/28, refcount now 2
01:21:06:  Dropping refcount on 192.168.5.96/28, refcount now 2
01:21:06: Packetizing timer expired on Serial0
01:21:06: Packets pending on Serial0
01:21:06: Intf Serial0 packetized QUERY 16-17
01:21:06:  Interface is now quiescent
01:21:06: Building multicast QUERY packet for Serial0, serno 16-17
01:21:06:  Items: U16 U17
01:21:06: Packetizing timer expired on Ethernet1
01:21:06: Packets pending on Ethernet1
01:21:06: Intf Ethernet1 packetized QUERY 16-17
01:21:06:  Interface is now quiescent
01:21:06: Building multicast QUERY packet for Ethernet1, serno 16-17
01:21:06:  Items: 16 17
01:21:06: Packetizing timer expired on Serial1
01:21:06: Packet acked from 192.168.5.34 (Ethernet1), serno 16-17
01:21:06: Flow blocking cleared on Ethernet1
01:21:06: Multicast acked from Ethernet1, serno 16-17
01:21:06:  Found serno 16, refcount now 1
01:21:06:  Found serno 17, refcount now 1
01:21:06: Packet acked from 10.1.1.2 (Serial0), serno 16-17
01:21:06: Flow blocking cleared on Serial0
...
01:21:07: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial1,
  changed state to down
r1#show ip route
...
   192.168.5.0/24 is variably subnetted, 6 subnets, 2 masks
D    192.168.5.96/28 [90/40540160] via 192.168.5.34, 00:00:12, Ethernet1
D    192.168.5.64/28 [90/40537600] via 192.168.5.34, 00:32:10, Ethernet1
C    192.168.5.32/28 is directly connected, Ethernet1
D    192.168.5.48/28 [90/40537600] via 192.168.5.34, 00:32:10, Ethernet1
D    192.168.5.0/24 is a summary, 00:33:44, Null0
C    192.168.5.16/28 is directly connected, Ethernet0
   10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
D    10.0.0.0/8 is a summary, 00:32:11, Null0
C    10.1.1.0/24 is directly connected, Serial0
01:21:38: %LINK-3-UPDOWN: Interface Serial1, changed state to up
01:21:38: DNDB UPDATE 192.168.5.80/28, serno 0 to 20, refcount 2
01:21:38:  Anchoring Serial0, starting Serial0 timer
01:21:38:  Anchoring Ethernet1, starting Ethernet1 timer
01:21:38: Packetizing timer expired on Serial0
01:21:38: Packets pending on Serial0
01:21:38: Intf Serial0 packetized UPDATE 20-20
01:21:38:  Interface is now quiescent
01:21:38: Building multicast UPDATE packet for Serial0, serno 20-20
01:21:38:  Items: S20
01:21:38:  Suppressed.
...
01:21:38: Building startup packet for 192.168.5.82, serno 1-20
01:21:38:  Items: 1 2 S4 S5 6 7 8 19 S20
01:21:39: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial1,
  changed state to up
01:21:40: Building startup packet for 192.168.5.82, serno 1-20
01:21:40:  Items: 1 2 S4 S5 6 7 8 19 S20
01:21:42: DNDB UPDATE 192.168.5.96/28, serno 19 to 21, refcount 3
01:21:42:  Anchoring Serial1, starting Serial1 timer
01:21:42:  Anchoring Ethernet1, starting Ethernet1 timer
01:21:42:  Anchoring Serial0, starting Serial0 timer
01:21:42: Packetizing timer expired on Serial1
01:21:42: Packets pending on Serial1
01:21:42: Intf Serial1 packetized UPDATE 21-21
r1#show ip route
...
   192.168.5.0/24 is variably subnetted, 7 subnets, 2 masks
D    192.168.5.96/28 [90/40514560] via 192.168.5.82, 00:00:04, Serial1
D    192.168.5.64/28 [90/40537600] via 192.168.5.34, 00:00:04, Ethernet1
C    192.168.5.80/28 is directly connected, Serial1
C    192.168.5.32/28 is directly connected, Ethernet1
D    192.168.5.48/28 [90/40537600] via 192.168.5.34, 00:00:04, Ethernet1
D    192.168.5.0/24 is a summary, 00:34:12, Null0
C    192.168.5.16/28 is directly connected, Ethernet0
   10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
D    10.0.0.0/8 is a summary, 00:00:04, Null0
C    10.1.1.0/24 is directly connected, Serial0
r1#undebug all

Example 3-42 Troubleshooting Using EIGRP Show Commands

r1#show ip eigrp ?
 interfaces IP-EIGRP interfaces
 neighbors  IP-EIGRP neighbors
 topology  IP-EIGRP Topology Table
 traffic   IP-EIGRP Traffic Statistics
r1#show ip eigrp neighbors
IP-EIGRP neighbors for process 500
H  Address         Interface  Hold Uptime  SRTT  RTO Q Seq
                    (sec)     (ms)    Cnt Num
1  192.168.5.82      Se1      12 00:01:15  24 2280 0 38
2  10.1.1.2          Se0      14 00:33:05  44 2280 0 3
0  192.168.5.34      Et1      10 00:34:23  25  200 0 35
r1#show ip eigrp interfaces
IP-EIGRP interfaces for process 500
          Xmit Queue  Mean  Pacing Time  Multicast  Pending
Interface  Peers Un/Reliable SRTT  Un/Reliable  Flow Timer  Routes
Et0      0    0/0     0    0/10      0        0
Et1      1    0/0    25    0/10      108      0
Se1      1    0/0    24    10/380    380      0
Se0      1    0/0    44    10/380    556      0
r1#show ip eigrp topology ?
 <1-65535>    AS Number
 A.B.C.D     Network to display information about
 active      Show only active entries
 all-links    Show all links in topology table
 pending     Show only entries pending transmission
 summary     Show a summary of the topology table
 zero-successors Show only zero successor entries
 <cr>
r1#show ip eigrp topology active
IP-EIGRP Topology Table for process 500

The next two commands are very useful for EIGRP troubleshooting. Turn them on and note the output (as I do in Example 3-43). To force some changes, I issued a no shut on r1s1 to see a neighbor change. I brought that interface back up, turned off the neighbor changes, and turned on the neighbor warnings. Then I went to r2 to bounce (shut/no shut) the Ethernet 0 interface to observe the results.

Example 3-43 EIGRP Logging

r1(config)#router eigrp 500
r1(config-router)#eigrp ?
 log-neighbor-changes  Enable/Disable IP-EIGRP neighbor logging
 log-neighbor-warnings Enable/Disable IP-EIGRP neighbor warnings
r1(config-router)#eigrp log-neighbor-changes
r1(config-router)#end
...
r1(config)#interface s1
r1(config-if)#shut
01:42:34: %DUAL-5-NBRCHANGE: IP-EIGRP 500: Neighbor 192.168.5.82 (Serial1) is
  down: interface down
01:42:36: %LINK-5-CHANGED: Interface Serial1, changed state to
  administratively down
01:42:37: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial1,
  changed state to down
r1(config-if)#no shut
01:42:54: %LINK-3-UPDOWN: Interface Serial1, changed state to up
01:42:55: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial1,
  changed state to up
01:42:56: %DUAL-5-NBRCHANGE: IP-EIGRP 500: Neighbor 192.168.5.82 (Serial1) is
  up: new adjacency
r1(config-if)#exit
r1(config)#router eigrp 500
r1(config-router)#no eigrp log-neighbor-changes
r1(config-router)#eigrp log-neighbor-warnings
r1(config-router)#exit
r1(config)#interface s1
r1(config-if)#shut
01:43:33: %DUAL-5-NBRCHANGE: IP-EIGRP 500: Neighbor 192.168.5.82 (Serial1) is
  down: interface down
01:43:35: %LINK-5-CHANGED: Interface Serial1, changed state to
  administratively down
01:43:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial1,
  changed state to down
r1(config-if)#no shut
r2(config)#interface e0
r2(config-if)#shut
r1 01:44:30: %DUAL-5-NBRCHANGE: IP-EIGRP 500: Neighbor 192.168.5.34 (Ethernet1) is
  down: holding time expired
r2(config-if)#no shut
r1 01:44:55: %DUAL-5-NBRCHANGE: IP-EIGRP 500: Neighbor 192.168.5.34 (Ethernet1) is
  up: new adjacency

Look back through the examples and compare the output of some of the commands to your drawing. There are still issues with EIGRP. Can you spot them? Fix them and continue on to the next Trouble Ticket.

A simple ping test or looking at the routing tables for network 10.0.0.0 should have told you that EIGRP automatically summarizes on the classful boundary. On r1 and r5, for example, you could see 10.1.1.0 but not 10.2.2.0. r3 and r4 displayed 10.2.2.0, but not 10.1.1.0. The fix is in Example 3-44, and the commands are required on r1 and on r3 since they cross network boundaries. Test by displaying the routing table on r2 and ping testing from router to router, although a better test would be from hosta to 10.2.2.2.

Example 3-44 EIGRP no auto-summary Fix

r2#show ip route
   192.168.5.0/28 is subnetted, 6 subnets
D    192.168.5.96 [90/40514560] via 192.168.5.66, 00:03:35, Serial0
           [90/40514560] via 192.168.5.50, 00:03:35, Serial1
C    192.168.5.64 is directly connected, Serial0
D    192.168.5.80 [90/40537600] via 192.168.5.33, 00:03:36, Ethernet0
C    192.168.5.32 is directly connected, Ethernet0
C    192.168.5.48 is directly connected, Serial1
D    192.168.5.16 [90/307200] via 192.168.5.33, 00:03:36, Ethernet0
D  10.0.0.0/8 [90/40537600] via 192.168.5.33, 00:03:36, Ethernet0
r1(config)#router eigrp 500
r1(config-router)#no auto-summary
r1(config-router)#end
r1#copy running-config startup-config
r3(config)#router eigrp 500
r3(config-router)#no auto-summary
r3(config-router)#end
r3#copy running-config startup-config
r2>show ip route
Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP
    D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
    N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
    E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
    i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, * - candidate default
    U - per-user static route, o - ODR
Gateway of last resort is not set
   192.168.5.0/28 is subnetted, 6 subnets
D    192.168.5.96 [90/40514560] via 192.168.5.66, 00:00:50, Serial0
           [90/40514560] via 192.168.5.50, 00:00:50, Serial1
C    192.168.5.64 is directly connected, Serial0
D    192.168.5.80 [90/40537600] via 192.168.5.33, 00:00:50, Ethernet0
C    192.168.5.32 is directly connected, Ethernet0
C    192.168.5.48 is directly connected, Serial1
D    192.168.5.16 [90/307200] via 192.168.5.33, 00:00:50, Ethernet0
   10.0.0.0/24 is subnetted, 2 subnets
D    10.2.2.0 [90/41024000] via 192.168.5.66, 00:00:50, Serial0
         [90/41024000] via 192.168.5.50, 00:00:50, Serial1
D    10.1.1.0 [90/40537600] via 192.168.5.33, 00:00:51, Ethernet0
r2>

Notice how the 10.1.1.0 and 10.2.2.0 subnets now display in the routing table with a D in front. Just as with the previous RIPv2 examples, the no auto-summary command is critical in a situation such as this in order for EIGRP to support discontiguous subnets. EIGRP enables you to summarize manually on any boundary and interface you so choose.

Trouble Ticket 5 Solution

You should now be able to successfully ping and trace between all your routers. That is great, but the real goal is host-to-host, end-to-end connectivity. See whether you can ping and tracert from hosta to hostc and capture the results. Of course, if you are like me, you may have forgotten to change the IP address information on your hosts. Obviously that is a prerequisite to the rest of this. Table 3-12 shows my host parameters. If you need more help with addresses, masks, and gateways, experiment a little more here by changing one at the time and observing the results. Practice does make perfect! Compare your results to my output in Example 3-45 and Figure 3-35. Also take time to examine how Sniffer displays EIGRP. Remember to save all your configuration files to the TFTP server with the rx-eigrp name format.

Figure 3-35 EIGRP and Client Initialization Traffic

Table 3-12 Trouble Ticket Host Configuration

	IP Address	Subnet Mask	Gateway
hosta	192.168.5.18	255.255.255.240	192.168.5.17
hostb	192.168.5.19	255.255.255.240	192.168.5.17
hostc	192.168.5.98	255.255.255.240	192.168.5.97

Example 3-45 Trouble Ticket 5 Ping and Trace Output

C:\>ping 192.168.5.98
Pinging 192.168.5.98 with 32 bytes of data:
Request timed out.
Reply from 192.168.5.98: bytes=32 time=20ms TTL=126
Reply from 192.168.5.98: bytes=32 time=10ms TTL=126
Reply from 192.168.5.98: bytes=32 time=10ms TTL=126
Ping statistics for 192.168.5.98:
  Packets: Sent = 4, Received = 3, Lost = 1 (25% loss),
Approximate round trip times in milli-seconds:
  Minimum = 10ms, Maximum = 20ms, Average = 10ms
C:\>tracert 192.168.5.98
Tracing route to HOSTC [192.168.5.98]
over a maximum of 30 hops:
 1  <10 ms  10 ms  <10 ms 192.168.5.17
 2  20 ms  20 ms  20 ms 192.168.5.82
 3  20 ms  30 ms  30 ms HOSTC [192.168.5.98]
Trace complete.

I captured my file just after I made the change to the hostb IP changes. I started Sniffer upon bootup to capture the Windows 98 client bootup for later review. The file is chapter 3 eigrp and client initialization traffic sniffer capture.

The first couple of hundred lines of the Sniffer capture include hostb booting up to the Windows 98 operating system. Starting in line 221, I issued a ping and then a tracert from hosta to hostc as the output in Example 3-45 depicts. See the EIGRP multicast 224.0.0.10 hello address and IP handing off to EIGRP over protocol number 88.

You have completed the chapter Trouble Tickets when you feel comfortable with the tasks assigned and the various scenarios throughout the chapter. I saved my configurations to the TFTP server and also logged the output of show running-config to a file. I did not bother with changing the hosts files, but you certainly can if you so desire. However, I will have you write erase in the next chapter anyway.

Review or experiment in the areas where you need more help. Understanding and troubleshooting in a simple environment is certainly the foundation for understanding and troubleshooting complex protocols and technologies. Check your understanding with the chapter review questions.

Review Questions

Use this chapter and your practical troubleshooting knowledge and skills to answer the following questions. The answers are located in Appendix A, "Answers to Review Questions."

1. In the RIP scenario, why were you successful with using RIPv2 rather than RIPv1?

2. A Cisco router maintains ARP entries much longer than most PCs. How can you remove all entries from the ARP cache on a Cisco router? It would be less detrimental to all to just remove an entry associated with a given interface. Can you do that on a router? On a Windows-based PC?

3. Draw a table comparing TCP/IP layers, protocols, applications, and utilities to the OSI model.

4. On a Cisco router, show ip route displays the routing table. What are the numbers in brackets []?

5. Subnetting, aggregation, VLSM, CIDR, supernetting, and summarization are all about moving bit boundaries. Which one(s) move the network mask bit boundary to the right?

6. Assume you moved into apartment 172.16.3.10 (host address) located at 172.16 Broad Creek Drive (network address). Other floors in the apartment building are numbered 172.16.1.0, 172.16.2.0, and 172.16.4.0. What floor (subnet) are you on? What are all the available hosts on that subnet? What is the directed broadcast address of your subnet?

7. Compare the protocol and port numbers for telnet, RIP, FTP, and TFTP.

8. You can ping by the IP address but not by the hostname. What is a very likely problem?

9. You need to forward DHCP requests to another subnet, but you do not want to forward NetBIOS communications. Is this possible?

10. What IOS command enables you to verify that RIP sends broadcast routing updates? To what address are broadcast updates sent?

11. Using 192.168.5.0/24, address the network according to the following requirements: three LAN segments—one with 125 hosts, one with 50 hosts, and one with 25 hosts—and at least two and maybe more WAN segments.

12. You are having a problem with three subnets connected via two Cisco routers. Each router can ping its own interfaces but can't get to the far side of the other router. So you decide to put in the appropriate default route statement, but things still are not operational. You are not running routing protocols because default routes serve this scenario well. Can you spot the issue?

Summary

Shooting trouble with IP is not just crossing the bridge to get to the beach. As more and more companies are adopting IP-based networks, network engineers and analysts must continue to ensure the internetworks are available, reliable, redundant, responsive, accessible, and secure. Every one of you must understand TCP/IP end-to-end issues and know where to find the right tools and use the proper methods to make the IP world happy. This chapter reviewed IP protocols and packets, addressing, and routing protocol topics to get you started with IP troubleshooting. The next chapter examines IPX and many other hidden troubleshooting issues. Then the focus turns to Layer 2 LAN and WAN technologies to continue to build your practical troubleshooting skills.