Home > Articles > Cisco Network Technology > General Networking > Cisco ISP Software and Router Management

Cisco ISP Software and Router Management

Chapter Description

This chapter covers general features that ISPs should consider for their routers and network implementations. Most are good design practices and don't leverage particular unique Cisco IOS Software features, but each demonstrates how IOS Software can aid the smooth operation of an ISP's business.

From the Book

Cisco ISP Essentials

Cisco ISP Essentials



Enabling NetFlow on routers provides network administrators with access to packet flow information from their network. Exported NetFlow data can be used for a variety of purposes, including security monitoring, network management, capacity planning (as in Figure 2-1), customer billing, and Internet traffic flow analysis.

Figure 5Figure 2-1 Netflow in Its Capacity-Planning Role

NetFlow is available on all router platforms from the 2600 series upward from the 12.0 software release onward. It was first introduced in 11.1CC on the 7200 and 7500 platforms. It can be enabled on a per-interface basis on the routers, as in the following example:

interface serial 5/0
 ip route-cache flow

If CEF is not configured on the router, this turns off the existing switching path on the router and enables NetFlow switching (basically modified optimum switching). If CEF is configured on the router, NetFlow simply becomes a "flow information gatherer" and feature accelerator—CEF remains operational as the underlying switching process.

NetFlow Feature Acceleration

NetFlow feature acceleration works for a limited set of features that can take advantage of flow process short cuts. NFFA reserves space in the flow cache for state information belonging to features converted to use the flow acceleration. The features can then attach per-flow state to the cache entry, using NetFlow as a quick way to access information that is flow-based. For example, NetFlow policy routing (NPR) uses flow acceleration to eliminate route-map checks on a per-packet basis. NetFlow feature acceleration is turned on with the following command:

ip flow-cache feature-accelerate

As of 12.0(11)S, the following features have been converted to work with NetFlow feature acceleration:

  • Numbered access lists
  • Named access lists
  • IP accounting
  • Crypto decrypt
  • Crypto encrypt
  • Policy routing
  • WCCP redirection

NetFlow Statistics—Basics

To view NetFlow information on the router, simply enter the command show ip cache flow. This displays the current flow cache on the terminal screen (see Example 2-1).

Example 2-1 Sample Output from Displaying Flow Information on a NetFlow-Enabled Router

gw>sh ip cache flow
IP packet size distribution (410772243 total packets):
  1-32  64  96 128 160 192 224 256 288 320 352 384 416 448 480
  .000 .168 .384 .102 .160 .107 .019 .005 .003 .001 .001 .000 .000 .000
  512 544 576 1024 1536 2048 2560 3072 3584 4096 4608
  .001 .000 .035 .000 .003 .000 .000 .000 .000 .000 .000
IP Flow Switching Cache, 4456704 bytes
 15074 active, 50462 inactive, 125120769 added
 369493980 ager polls, 0 flow alloc failures
 last clearing of statistics 4d05h
Protocol     Total  Flows  Packets Bytes Packets Active(Sec) Idle(Sec)
--------     Flows   /Sec    /Flow  /Pkt    /Sec     /Flow     /Flow
TCP-Telnet     605    0.0       44    52     0.0       8.1       9.1
TCP-FTP       3494    0.0       22    64     0.2       9.4      12.9
TCP-FTPD      4104    0.0      757   376     8.4      34.9       5.7
TCP-WWW     845158    2.3       16   281    39.1       4.5       6.8
TCP-SMTP     87119    0.2       10   201     2.5       4.2      13.1
TCP-X           59    0.0        2    68     0.0       0.4      12.0
TCP-BGP      62074    0.1        5   255     0.9       9.6      18.5
TCP-NNTP         5    0.0        3    48     0.0       8.8      19.6
TCP-Frag         2    0.0        2    40     0.0       0.1      21.2
TCP-other 11879955   32.3        5   141   174.2       2.5       7.5
UDP-DNS   70078211  191.0        3    90   586.3       4.8      19.1
UDP-NTP      31804    0.0        1    72     0.0       0.0      19.0
UDP-TFTP       327    0.0        3   153     0.0       4.8      19.2
UDP-Frag         9    0.0        4   311     0.0      22.5      18.2
UDP-other 41601240  113.4        2   157   301.3       4.1      19.1
ICMP        498404    1.3        4   170     5.7      10.7      19.0
IGMP             2    0.0      113   551     0.0       6.8      19.8
IP-other     20236    0.0        4   299     0.2      12.7      18.7
Total:   125112808  341.1        3   126  1119.2       4.4      17.9
SrcIf     SrcIPaddress    DstIf     DstIPaddress    Pr SrcP DstP  Pkts
Se2/0  Fa1/0  11 2245 0035     1
Fa1/0  Se2/0  11 0035 2245     1
Fa1/0   Se2/0 11 0035 0C1C     1
Se2/0 Fa1/0   11 0C1C 0035     1
Se2/0  Fa1/0   11 0408 0035     1
Fa1/0   Se2/0  11 0035 0408     1
Fa1/0   Se2/0    11 0035 0035     2
Se2/0  Fa1/0   11 0A6B 0035   248
Se2/0    Fa1/0   11 0035 0035     4
Fa1/0   Se2/0    11 0035 0035     4
Fa1/0   Se2/0  11 0035 0A6B   248
Se2/0    Fa1/0   11 0035 0035     2
Se2/0    Fa1/0   11 B034 0035     2
Fa1/0   Se2/0    11 0035 B034     2

The first part of the output displays the packet size distribution of the traffic flowing into the interfaces that NetFlow is configured on. The next portion of the output displays the flows, packet size, activity, and so on for the flows per well-known protocol. The final section displays the source and destination interfaces/addresses/ports for the currently active traffic flows.

It is also possible to export this collected data to a system that will collect the data, allowing the ISP to carry out further analysis. Public-domain software is available (cflowd from Caida and NetFlowMet from the University of Auckland, for example), as well as fully featured and supported commercial products, such as Cisco's NetFlow Collector and Analyzer packages.

NetFlow Data Export

The greatest benefits of NetFlow are found when its data is exported to collection systems and then are analyzed and processed. Cisco has adopted a broad approach to facilitate this activity. These include donations for freeware collection/analysis software, Cisco's own commercial software, tools for others to create their own software, and partnerships with companies that make commercial-grade billing systems based on NetFlow export.

To export the data, the following configuration commands are required:

ip flow-export version 5 [origin-as|peer-as]
ip flow-export destination x.x.x.x udp-port
ip flow-export source Loopback0

The first command line sets the export version to 5 (basically this includes BGP information such as AS number) and has options to include origin-as or peer-as in the exported records. Most ISPs use the origin-as option because that will record the origin AS of the prefix originating the flow. This has become a frequently asked question on the CAIDA cflowd list, with ISPs forgetting the origin-as option and then not understanding why so many of their exported records have an origin of AS 0.

The second command line configures the IP address of the destination system, the NetFlow collector system, and the UDP port that the collector is listening on. Most ISPs use high UDP ports, such as 9999 or in the 60,000s. Note that because the flow records use UDP, it is important to design the infrastructure so that the flow collector is not too far away from the originating router. Some ISPs that use NetFlow for billing purposes build a separate management network simply to support this function.

The third command line originates all the flow traffic using the IP address of the loopback interface. This makes the cflowd configuration file easier to construct for several routers because most ISPs number their router loopbacks out of one contiguous block.

To determine the status of the flow export, it is possible to check on the router to see what has been sent. Obviously the collector system should be checked as well—cflowd, for example, has extensive instructions on how to debug any flow export problems. An example of the usage of the IOS Software command follows:

gw>sh ip flow export

Flow export is enabled

Exporting flows to (9998)

Exporting using source interface Loopback0

Version 5 flow records, origin-as

264038749 flows exported in 8801292 udp datagrams

0 flows failed due to lack of export packet

6079835 export packets were sent up to process level

0 export packets were punted to the RP

0 export packets were dropped due to no fib

0 export packets were dropped due to adjacency issues

0 export packets were dropped due to fragmentation failures

0 export packets were dropped due to encapsulation fixup failures

A new feature as of Cisco IOS Software release 12.0(5)S is NetFlow aggregation, in which summarization/aggregation of the flow records is carried out on the router before the data is exported to the collecting system. The aim is to reduce the amount of data going across the network from router to flow collector, thereby improving the reliability of the collecting system. Flow aggregation is enabled by the following commands:

ip flow-aggregation cache as|destination-prefix|prefix|protocol-port|source-prefix


export destination x.x.x.x UDP-port

Subcommands required include enabled, which switches on the flow aggregation, and export destination, which lists the host that will gather the aggregated records. The collector host needs to support NetFlow Type 8 records to be capable of reading the aggregated information.

6. Turn On Nagle | Next Section Previous Section