Cisco Unified Contact Center Enterprise Platform Deployment

Date: Jul 27, 2011 By Gary Ford. Sample Chapter is provided courtesy of Cisco Press.
This chapter covers the steps needed for planning a Cisco Unified Contact Center Enterprise (UCCE) deployment, installing UCCE software, and testing the deployment.

This chapter covers the following subjects:

  • Planning for a UCCE deployment
  • UCCE software installation
  • Deployment testing

The initial deployment and installation of a Cisco Unified Contact Center Enterprise (UCCE) platform is only a small part of the total ownership and administration required to maintain the solution. However, getting the installation correct is an important step to ensuring that the solution remains supportable and problem-free.

As you discover in this chapter, the deployment of an advanced solution such as UCCE can be a long process requiring a dedicated project team with experienced technical specialists.

This chapter briefly introduces the Cisco Lifecycle Service methodology used by Cisco and its partners to deploy advanced unified communications solutions. Although the lifecycle covers the entire deployment process from the initial business ideas through to the solutions optimization, this chapter focuses on the design and implementation of the core UCCE software components that need to be in place before the application-level business requirements can be implemented. Chapter 7, "UCCE Application Configuration," details the aspects of application-level configuration.

It is common practice to split larger UCCE installations among various deployment engineers or several small teams of engineers. This is done because a complete UCCE deployment encompasses several different products or platforms that require specialist skills. The different technical skill sets required for a UCCE deployment include the following:

  • Business analyst: The business analyst is responsible for converting the business requirements into detailed technical requirements. The business analyst might also be required to collect the end-user data. This includes details such as the agent names, IDs, skill groups, teams, extension numbers, and locations. Solutions architect: Having the responsibility for the overall solution, the solutions architect is aware of the "bigger picture" of the technical aspects and understands all the integration interfaces of how the different components interact.
  • Microsoft Windows specialist: Many organizations prefer to deploy their own MS Windows architecture for UCCE because of its integration with Windows Active Directory. If the customer prefers that the Cisco partner perform the Windows and SQL Server installations, a specialist is then required with these skills.
  • Unified CM engineer: Responsible for the installation and configuration of the Unified CM cluster, including voice gateways and interfaces to the PSTN or any legacy time-division multiplexing (TDM) technologies.
  • Unified IVR/CVP engineer: The Interactive Voice Response (IVR) engineer is responsible for the installation and configuration of the chosen IVR platform and then the subsequent integration of the IVR into UCCE.
  • UCCE installation engineer and application engineer: Both of these roles can be performed by the same engineer. This is especially true for smaller deployments. For large installations that have hundreds of complex routing scripts, skill groups, and thousands of agents, these roles are usually split to ensure that the project is delivered on time. In practice, the UCCE installation engineer is also often skilled at IVR installations, so he can perform the initial installation of the core UCCE components and then hand over the application configuration to another engineer while continuing with the IVR work in parallel.
  • Third-party specialist: Several products exist outside of the core components that integrate into UCCE including voice recording, workforce management, speech recognition, and email/web collaboration. These products often require specialist installations by skilled engineers or the product vendors.

Lifecycle Services Approach

To instigate a consistent approach to network implementations, Cisco created a network lifecycle methodology that partners and end customers could use throughout to achieve their network-related business goals. As organizations began to deploy unified communications technologies, many people experienced challenges with implementing UC technology on their existing network infrastructure. Many of these challenges were because of a lack of a structured approach.

Cisco recommends that a proven lifecycle approach is followed when deploying complex solutions. When an organization engages with Cisco and a Cisco partner for the deployment of a UCCE solution, the Cisco consulting engineers use the Cisco Lifecycle Services approach to ensure smooth project delivery.

The Cisco Lifecycle approach is built to the standard of the Information Technology Infrastructure Library (ITIL) and other standards-based frameworks. The approach, shown in Figure 6-1, comprises six distinct phases, collectively known as the Cisco PPDIOO Lifecycle Methodology.

Figure 6-1

Figure 6-1 Cisco PPDIOO Lifecycle Methodology

The aim of the lifecycle is to align the business and technical requirements at each phase:

  • Prepare: In the prepare phase of the lifecycle, an organization determines a business case and financial rationale to support the adoption, enhancement, or migration to the new technology. The organization is required to develop a technology strategy and identify a technology and a high-level architecture to meet those requirements. After the financial and business value of the chosen technology has been assessed, the company can validate the technology's features and functionality through proof-of-concept testing.
  • Plan: Successful deployment depends on an accurate assessment of an organization's current network and overall readiness to support the proposed technology. In the plan phase, a company determines whether it has adequate resources to manage a technology deployment project to completion. Certainly for UCCE deployments, a Cisco Authorized Technology Provider (ATP) partner is required for deployment, so this planning phase would almost certainly be a joint task between the organization and the partner. A detailed project plan is created to identify resources, potential difficulties, individual responsibilities, and critical tasks necessary to deliver the final project on time and on budget.
  • Design: During the design phase, a comprehensive and detailed design is created to meet the business and technical requirements. A design aligned with business goals and technical requirements can improve network and platform performance while supporting high availability, reliability, security, and scalability. Many of the features deployed for a UCCE solution are out-of-the-box components that require configuring to suit the organization, but UCCE also has several programming interfaces that can be developed against to provide the organization with enhanced, custom features. These should also be documented during the design phase. The design phase can also guide and accelerate successful implementation with a plan to stage, configure, test, and validate network operations.
  • Implement: In the implementation phase, an organization works to integrate the UCCE solution in accordance with the design—without compromising network availability or performance or impacting the current operation of any existing voice or contact center solutions. The implementation phase is probably the most visible phase to the organization as the project work takes progress. For a solution as large as UCCE that encompasses contact center, telephony, and networking components, the installation, configuring, integrating, testing, and commissioning of the systems usually requires a large team of engineers with a wide range of skill sets. After the solutions operation is validated, an organization can begin expanding and improving IT staff skills through professionally delivered training courses and on-the-job ad hoc training from the Cisco partner.
  • Operate: Day-to-day platform operations represent a significant portion of an IT budget. A key business driver is to reduce the total cost of ownership (TCO) of a solution's running costs while continually maintaining, and potentially enhancing, performance. Throughout the operate phase, an organization proactively monitors the health and performance of a solution to improve service quality, reduce disruptions, and maintain high availability and reliability. Contact centers typically have a higher staff turnover rate than regular office-based roles. This has the impact that a large portion of work will be performing adds, moves, and changes.
  • Optimize: Contact centers are generally the first, and sometimes only, point of contact into the enterprise for an individual customer. Contact centers strive to be the best and to provide a high standard of customer service to their callers. Business best practices and contact center technology are constantly evolving to ensure that the caller is given a good experience. In the optimize phase, an organization is continually looking for ways to achieve operational excellence through improved performance and expanded services. This results in the changes to business requirements and the potential to implement new technology. In recent years, contact center technology has advanced so rapidly that occasionally technology drives through business change. Many new features available in recent versions of UCCE have resulted in organizations wanting to upgrade as they can see a direct competitive advantage for implementing the new solutions.

Prepare and Plan

The preparation and planning phases for a UCCE deployment are outside the scope of this book. To achieve success in these phases, it is necessary to engage with a Cisco partner capable of delivering a solution that meets the organization's vision and requirements.

Cisco has a vast product and services catalogue covering a wide range of technologies. It is therefore important that the chosen Cisco partner has a proven background for the planned deployment. To assist with choosing the correct partner, Cisco requires that partners specialize in certain technology areas and prove their capabilities by having an agreed-upon number of trained and qualified staff with access to UCCE laboratory facilities.

For a Cisco partner to deploy and support a UCCE solution, it must have achieved the Authorized Technology Provider (ATP) accreditation for UCCE. The ATP program is an invitation-only program focused on the high-end enterprise contact center marketplace. Only when the partner has satisfied the entry criteria can it then resell and support UCCE solutions.

Design

With the different deployment models available and the various options about the distribution of software components based on the size of the contact center, you can design a UCCE solution with many permutations. However, you needd to consider several key points when creating a design:

  • Scalability: Is the solution sized correctly for the number of staff that will use the platform initially and for at least 1 year following going live?
  • Survivability: Does the proposed solution take advantage of the multiple levels of redundancy available with UCCE?
  • Compliance: Does the design meet or exceed the customer's business requirements? Are the hardware and software compatible with the bill of materials, the Solution Reference Network Design (SRND) Guide for UCCE, and the Cisco Assessment to Quality (A2Q) process?

Software Versions

When creating a UCCE design, one of the initial considerations is which version of UCCE should be used. It would be easy to insist that the customer deploys the latest version of UCCE. However, many customers prefer to wait for at least the first maintenance release of a product to become available. It is also wrong to select a software version in isolation. A key feature of unified communications is the capability to integrate with many different products or platforms. It is therefore important to consider the following before making a decision on which version to choose:

  • Does the chosen software version have the required features? Most new features are introduced in the major software versions; however, some features do become available in the minor releases.
  • Is the software version first customer shipment (FCS), or has it been available for some time and is known to be a stable, or preferred, release? Are several maintenance releases available that have fixed the majority of known bugs? Good practice is to check the release notes for any known outstanding bugs. Many of these bugs are minor, but is there anything in particular that can affect this specific customer?
  • Are specific IOS versions required on the network infrastructure, in particular, voice gateways, CUBE routers and analog interfaces?
  • Are there any specific IOS features that need to be enabled on the network, including QoS, firewall ports, and traffic engineering?
  • Does a network management requirement exist to ensure that operational support personnel are aware of issues in real time?
  • Does the organization already have a Cisco Unified CM platform? Is this software version compatible, or is an upgrade required?
  • Which other Cisco Unified Communications products are proposed or are already in use? This includes IVR, voicemail, and Unified Communicator.
  • Does the organization require the integration of any third-party products such as voice recording, wallboards, customer relationship management (CRM) integration, attendant consoles, and instant messaging?
  • Is any legacy TDM integration required?

As detailed in Chapter 3, "Deployment Models," Cisco produces a compatibility matrix to assist when choosing software versions. Two versions of the matrix are available: a generic Unified Communications matrix that details product compatibility with Unified CM and a contact center—specific matrix for products including Customer Voice Portal (CVP) and IP IVR.

Cisco also regularly performs an IP communications systems test. This is a standard methodology for Cisco to perform systemwide testing of all Unified Communications products in a single laboratory environment. A major deliverable of this testing is a recommendation of compatible software releases that have been verified during this testing process. Organizations that plan to deploy multiple voice applications and infrastructure products can adopt these recommendations in their design.

Platform Sizing

When designing a Cisco Unified Communications solution, an essential application for the solutions architect to work through is the Unified Communications Sizing Tool (see Figure 6-2). This tool enables the design team to step through a comprehensive set of questions to specify the majority of components (CVP, IP IVR, Expert Advisor, Agent Desktop software) that make up the UC solution. As well as detailing the software components, the tool enables the design team to enter details including call-handling parameters, software versions, anticipated service levels, and redundancy options.

Figure 6-2

Figure 6-2 Unified Communications Sizing Tool

The tool prompts the user with a large number of design questions and can take considerable effort to complete correctly. The resulting output of completing the sizing tool is an Adobe PDF document detailing design options and performance metrics based on the data entered. This document should also be submitted to the A2Q process.

The sizing tool can also be used for designing expansions to existing UCCE platforms.

Platform Redundancy

You can design redundancy into a UCCE solution in many different ways. The recommended methods are as follows:

  • Hardware redundancy within a component: Examples of this include multiple disks within each server using a RAID mechanism or dual power supplies.
  • Node redundancy: This involves distributing the different nodes (loggers, routers, PGs) over multiple servers.
  • Distributed architecture: The UCCE software architecture is naturally redundant through its use of a Side A and Side B. It is common to separate the core platform over two physically diverse locations. The LAN and WAN connections are also diversely routed to separate the UCCE private and public network traffic.

Server Naming Conventions

Early versions of UICM did not have many of the supportability tools available today. When enabling trace settings, collecting logs, and performing administration duties, the platform required the support engineer to frequently establish remote control sessions to the required servers. To make it easier to identify the role of server, it felt that a standardized naming convention should be used. The commonly used server naming convention combined the customer instance name and the servers role. This naming convention was not mandatory, but many enterprises chose to implement it.

The naming convention used the format of three concatenated acronyms, GEOXXXYYYY, which are detailed in Table 6-1.

Table 6-1. Server Naming Convention

Acronym

Description

GEO

An abbreviation of GeoTel, the original developers of the UICM platform.

XXXX

An abbreviation of the customer name, usually the customer instance name.

YYYY

An acronym of the node type, as follows:

  • For a logger, use LGRA or LGRB, depending on the A or B side.
  • For a router, use RTRA or RTRB, depending on the A or B side.
  • For an administrative workstation, use AW proceeded by an integer value.
  • For a historic data server, use HDS preceded by an integer value.
  • For a peripheral gateway, use PGnA or PGnB, where n is the integer value of the PG as defined by its DMP identifier.

Table 6-2 provides an example of the server naming convention, with GEO being substituted for CSO (Cisco) and using a customer instance name of cus01.

Table 6-2. Example Server Naming for Customer Instance cus01

Server Name

Server Type

Node

Description

csocus01lgra

Logger

A

Logger Side A

csocus01lgrb

Logger

B

Logger Side B

csocus01rtra

Router

A

Router Side A

csocus01rtrb

Router

B

Router Side B

csocus01pg1a

PG

1A

Peripheral Gateway 1 Side A

csocus01pg1b

PG

1B

Peripheral Gateway 1 Side B

csocus01aw1

AW

1

Administrative Workstation 1

csocus01hds1

HDS

1

Historical Data Server 1

Unfortunately, the naming convention has not been continued in the later versions of the software, so standard acronyms for newer components such as the support tools server do not exist.

Deployment Spreadsheet

Solution design documents tend to focus on the architecture and the services that will be provided by the end solution, but they should also include the configuration settings that will be used. As UCCE is a distributed application that requires installation on several servers, the software setup process will be run many times, and potentially by different engineers, especially if the solution is distributed over several geographic locations.

Before deployment commences, it is advisable to have a node deployment spreadsheet that has been created by the solutions architect and reviewed and approved by the installation engineers.

The deployment spreadsheet is a quick reference guide with configuration details taken from the solution design that covers all the application settings to be used by the installation engineers during UCCE installation.

For simplicity, it is often easiest to represent the data in tabular form within a spreadsheet. Each sheet represents a single UCCE node and the settings required for the entire base software installation process.

Taking a logger installation as an example, the LoggerA sheet would require the configuration settings detailed in Table 6-3.

Table 6-3. LoggerA Installation Settings

Application

Setting

SQL Server

SQL Server version

SQL Server settings (database and log locations, sort order, service startup accounts)

SQL Server service pack version

UCCE Installation

Location and version of maintenance release (if required)

Drive destination for installation files

Install OS security hardening?

Install SQL Server security hardening?

A preshared key for Support Tools communication

UCCE Domain Manager (assuming LoggerA is the first UCCE node)

Customer instance name

Domain name

Customer instance number

Facility name

Domain usernames to assign to UCCE (Config, Setup, and WebView groups)

UCCE Web Setup

Administration username and password (that are assigned to the UCCE setup group)

NOTE: The italicized comments that follow are example answers for a LoggerA setup.

Deployment type [Enterprise]

Side [A]

Fault tolerance mode [Duplexed]

Router Side A private interface [csocus01rtrap]

Router Side B private interface [csocus01rtrbp]

Logger Side A private interface [csocus01lgrap]

Logger Side B private interface [csocus01lgrbp]

Enable historical/detail data replication [Y]

Display database purge configuration steps [N]

Enable outbound option [N]

Reboot on error [N]

Reboot on request [N]

Do not modify service account [selected]

Stop and then start the logger [N]

This same method would need to be applied to all the UCCE components, including routers, peripheral gateways (PG), support tools server, and IVRs to provide a comprehensive deployment spreadsheet. It is easy to see that creating these spreadsheets for an entire deployment can be time consuming, but doing so ensures that consistency is attained throughout the installation.

It is also advisable for the sheet to have a signoff section so that the installation can append a date and time that the configuration took place. This is useful for establishing a timeline in case retrospective changes need to be applied in the future. It also helps the project manager determine the project's progress.

Network Services

Network services are the foundation protocols and applications that run on the network to provide functionality to higher-layer applications and products. UCCE relies on several underlying network services.

DNS and HOSTS

The Domain Name System (DNS) service and HOSTS files provide device hostnames to IP address resolution. The communication between UCCE nodes is through IP. When configuring the various UCCE components, hostnames are generally used to provide support engineers with human-readable server names to make troubleshooting easier. For IP communication to take place between two servers, the server names need to be resolved to an IP address.

DNS services tend to be reliable, but the loss of the DNS service could have major consequences on the reliability of the UCCE platform. It is common practice to create HOSTS files that are a static list of server hostnames and their associated IP addresses. The HOSTS files usually list both the public and private addresses of only the UCCE server interfaces and are copied to each UCCE server. The servers also have details of the DNS servers configured on the public network interface controller (NIC) so that the UCCE server has access to other non-UCCE servers and services.

A disadvantage of using a HOSTS file is that the HOSTS file needs to be manually maintained and distributed throughout the necessary servers if any UCCE IP address or server names are changed, including the addition of new servers, such as PGs. Fortunately, the core servers that comprise a UCCE solution infrequently change.

Quality of Service

Network quality of service (QoS) is the ability to provide different priorities to streams of IP traffic for different applications, users, or data flows. QoS can be implemented in different ways. Usually the traffic is classified and marked on entry to the network. It is then prioritized as it traverses the network through the use of queuing or reservation algorithms. Both LAN and WAN traffic can be prioritized.

Although the core UCCE servers do not actually touch any voice streams, an entire UCCE solution comprises three different types of traffic:

  • Data traffic consists of general traffic between clients and servers, such as the communication or heartbeat traffic between the central controllers, Computer Telephony Integration (CTI) data sent to an agent desktop application, or database replication traffic.
  • Voice traffic consists of Real-Time Transport Protocol (RTP) packets that actually contain the packetized voice streams between IP phones, gateways, or IVRs.
  • Call control traffic consists of the protocols used to perform control functions such as the communication between a Unified CM subscriber and an IP IVR, or the SCCP control messages sent when an IP phone goes off-hook.

To understand QoS for UCCE, it is also necessary to understand the two independent communications networks used in a UCCE deployment.

Figure 6-3 shows the distributed components of a standard UCCE deployment. The central controllers and PGs each have two or more NICs. One NIC is connected to the public network (sometimes called the visible network), and the other is connected to the private network. The private network carries synchronization and heartbeat traffic. The public network carries all other traffic. Components such as the Unified CM servers and admin workstation historical data services (AW HDS) do not need a connection to the UCCE private network.

Figure 6-3

Figure 6-3 UCCE Independent Communications Networks

It is important to understand that the PG private network is different from the central controller's private network. The private interfaces for the PGs do not need to communicate with the private interfaces for the central controllers. This, however, does not mean that two separate private networks are required for architectures such as clustering over the WAN. In many cases, the two private networks are combined, but just sized correctly to support the necessary bandwidth requirements. For distributed deployments with both halves of the central controllers located on different physical sites, the private network does need to be physically separate from the public network. This often results in a private point-to-point link being installed purely for the private traffic. This diverse network is required so that two network connections exist between each side of the central controllers to ensure that the central controllers can still communicate and process calls in the event that one network connection fails. If both network connections fail, Side A or B attempts to continue based on an algorithm determined by the side of the router and the number of PGs with which the router is in communication. The requirement for the private link is often the most discussed point during the A2Q process!

For many smaller single-site deployments that combine the logger and router onto the same server, the private network is achieved by using a crossover cable between both machines.

The UCCE solution supports QoS tagging in the application. This means that the UCCE component marks the traffic with the assigned QoS classification as it leaves the application and therefore requires no further marking in the network. Assuming that the network trusts the QoS tagging, the QoS marking will be retained until it reaches its destination.

In UCCE version 8.5, QoS tagging is currently supported only for the private networks of the router and PG processes. Figure 6-4 shows an example of the default QoS tagging for Router A defined during web setup.

Figure 6-4

Figure 6-4 Router A QoS Marking for the Private Network

Prior to applications-based QoS marking (and for networks capable of only marking the packets at the network edge), UCCE uses additional IP addresses assigned to the public and private NICs. UCCE applications now actually use three traffic priorities—low, medium, and high—but prior to this, only two priorities were used. These priorities were based on the source and destination IP addresses. This allowed the marking and routing of traffic within the network rather than at the application. The traffic from these IP addresses also used specific TCP/UDP ports based on an algorithm that incorporates the instance number. This was so that hosted systems with multiple customer instances would not clash.

The IP addresses for the different interfaces (public/visible or private) and their priority (normal or high) were allocated to hostnames for easy configuration within the application. These names would also be detailed in the HOSTS file. Figure 6-5 shows a screen shot for a router setup that uses the visible, private, and high-priority private (termed private high) host names. Notice the p and ph at the end of the hostname to signify private and private high, respectively.

Figure 6-5

Figure 6-5 Router A Hostnames for the Private and Public Networks

Databases

An important aspect of designing a UCCE solution is to ensure that the databases are sized correctly. Database sizing has a direct impact on the amount of data that can be retained in all the databases.

The logger database is required to store all the configuration data and a limited amount of historical data. The HDS database stores all the long-term historical reporting data and call detail records. Many organizations want to retain at least 3 years worth of historical data so that they can analyze call trends over a long time period. Both the logger and the HDS databases are created by the installation engineer using the ICM Database Administrator (ICMDBA) tool, as shown in Figure 6-6.

Figure 6-6

Figure 6-6 The ICMDBA Tool Used for Creating the Logger and HDS Databases

The Administration and Data Server also has a configuration and real-time database, but this is not created with ICMDBA during installation; instead, the installer program automatically creates this database.

The server specification detailed in the UCCE bill of materials document typically specifies disk sizes that can easily meet the requirements of all but the largest contact centers. However, it is the responsibility of the designer and installation engineer to correctly size the database during installation. Unfortunately, the UCCE Sizing Tool does not provide any guideline sizes. You can use an ICM System Sizing Estimator tool to give database sizing approximations.

Figure 6-7 shows a screen shot of the Sizing Estimator running on an Administration and Data Server.

Figure 6-7

Figure 6-7 ICM System Sizing Estimator

The tool works by allowing the designer to enter approximate figures for the configuration data and the required retention periods. The tool then indicates an estimated required database size. As with all estimations, it is good practice to factor in an amount of anticipated growth.

Cisco A2Q Process

An important part of the design and ordering process is for the high-level solution to be approved by Cisco. Previously called Bid Assurance, the Assessment to Quality (A2Q) is a high-level design review of the proposed solution that takes place between the Cisco ATP partner responsible for deployment and the Cisco A2Q review team.

The A2Q process usually consists of the ATP partner submitting a series of documents to the review team. As the process is only a high-level review, the documents submitted include the following:

  • The A2Q questionnaire, which comprises a series of questions to give the review team the necessary background and design overview.
  • The bill of materials (BOM), which forms the kit list of the actual components to be ordered from Cisco to build the UCCE platform.
  • A network design showing all the network connectivity, physical site locations, bandwidth, and QoS.
  • A statement of work (SOW), which explains the deployment process and the teams, partners, and possibly Cisco professional service personal who can perform the deployment.

After the documents have been submitted, the review team schedules a conference call with the Cisco partner to discuss the design. The call typically lasts for approximately 30 minutes, and the conference participants include the following:

  • Representatives from the ATP partner including the technical project manager and solutions architect responsible for the design.
  • The A2Q review team composed of several Cisco engineers from the Contact Center Business Unit (CCBU) familiar with all technologies relevant to the design, including Unified Communications Manager, IVRs, and network infrastructure.
  • The Cisco sales engineer assigned to work with the ATP partner, who typically has a detailed knowledge of the partner's capabilities and the customer's requirements.

The A2Q conference calls are not open to the end customer.

During the conference call, the review team works through the submitted documents to confirm and validate various design items and the rationale behind them. The review team also seeks clarification on who will actually perform the deployment and the methodology that will be followed. Although the ATP partner is the direct interface into Cisco, it is common for large partners to subcontract work to smaller companies. The accountability of the solution is that of the ATP partner and not the subcontractor.

The A2Q process can take place at any point during the sales cycle. Cisco recommends that A2Q takes place as early as possible because the process is not just a formality to receive approval. A2Q is an important step in the deployment process, and without A2Q approval, the ATP partner cannot purchase the UCCE software and licenses required for deployment.

As discussed, the A2Q process is a high-level design review performed during a brief conference call. A detailed design review could take days or weeks. Items discussed on the call typically involve the following:

  • That the solution is sized correctly to meet the requirements, such as the number of Cisco Unified Communications Manager (Unified CM) servers to support a given number of agents, or the number of IVR ports to provide IVR resources for a defined number of busy-hour call attempts (BHCA).
  • That the correct kit list is ordered. Often the wrong part codes are listed in the bill of materials, or some items are accidentally missed.
  • That adequate and skilled resourcing has been scheduled to perform the deployment and that realistic time frames have been proposed.

The review team takes part in a large number of design reviews on a regular basis, so they have familiarity with a range of UCCE deployments. Because they see so many proposed designs, they can also offer comprehensive feedback and advice to the ATP partner on potential enhancements and ways to better the solution, perhaps even reducing the overall cost of deployment.

From the A2Q reviews I have taken part in, I would recommend the following points:

  • Clearly document and detail the private and public network connectivity. The survivability of the UCCE platform during a failure depends on the correct deployment of a segregated private and public network. With complex deployments such as clustering over the WAN and split peripheral gateways, it is important to demonstrate to the review team that the platform can still transmit heartbeat and synchronization traffic during events of failure.
  • Check the kit list thoroughly. Often the bill of materials (BOM) is produced early in the design process to provide the sales team with a figure to quote to the end customer. During the design process, the BOMs might change to suit new requirements. Be sure to include the correct agent license part codes, media kits, and the required support products such as Essential Operate Service (ESW) and the Unified Communications Software Subscription (UCSS).
  • Ensure that the proposed solution is sized correctly to meet just slightly more than the minimum requirements, but is not overengineered. Many of the customers I have worked with have expanded their platforms over time to add more agents. Designing a solution to meet only the bare minimum is a false economy and can lead to performance issues.

The A2Q process is not a guarantee that the solution will be error-free when it is deployed, but it is a review that adds great value to all designs and has been proven to ensure that the end customer receives a solution that is fit for purpose.

Implementation

To achieve success in the operate and optimize phases of the lifecycle, it is necessary to have a successful implementation as this is the phase in which the foundation is laid. A misconfigured or badly performed software installation will continually cause problems later in the platform's lifecycle.

Server Builds

Before commencing with an installation of the UCCE application, it is worth ensuring that the underlying Microsoft Windows and third-party applications are installed correctly. All the server components for a UCCE solution must be compliant with the bill of materials document for the software version being deployed. The following list comprises the key items that should be checked:

  • MS Windows version and correct service pack
  • MS SQL Server version and correct service pack
  • MS SQL Server binary sort order, Named Pipes enabled, and the correct client protocol order (1st Shared Memory, 2nd Named Pipes, 3rd TCP/IP)
  • Network cards configured with the correct IP addresses and binding order
  • Static routes defined on the servers to route private UCCE traffic out of the correct NICs
  • Additional Windows components installed (SNMP, WMI Windows Installer Provider)
  • Cisco third-party tools

Software Installation

UCCE versions prior to version 8.0 were installed and configured from a single setup.exe application from the installation media. The version 7.x installer would leave behind a version of the installer called icmsetup.exe in the c:\icm\bin directory so that this version of the installer could be run for future configuration changes or the addition/removal of other components.

With UCCE version 8.x, the UCCE software is still installed from a setup.exe application; however, the configuration of the UCCE nodes is now performed through a web-based setup process, making the installation and configuration a two-step process.

Not all the UCCE components currently use the new web setup. The ones that do are as follows:

  • Loggers
  • Routers (including NICs and network gateways for hosted or carrier-connected deployments)
  • Administration and data servers
  • Administration client
  • WebView

The following components do not use the new web setup:

  • Peripheral gateway
  • CTI server
  • CTI OS server

Figure 6-8 shows an example of configuration using the new web installer, whereas Figure 6-9 shows the installation/configuration of a PG, which in version 8.0 uses the traditional method.

Figure 6-8

Figure 6-8 New Web-Based Installer

Figure 6-9

Figure 6-9 Old-Style Installer Still Used for the PGs in UCCE 8.0

Installation Order

Chapter 11, "Nodes and Processes," examines all the different processes that run on a UCCE server, and you can observe a preferred startup order to minimize the number of errors observed. A similar order also exists when performing an initial installation.

Installing UCCE Logger A

The steps for installing UCCE Logger A are as follows:

  • Step 1. Run setup.exe and install the Logger A node. The setup.exe installer performs a base installation of the UCCE application onto the server. The installer checks for third-party applications such as SQL Server and prompts the engineer if they are missing or configured incorrectly. Include a maintenance release if required.
  • Step 2. Run DomainManager.exe on Logger A. This creates the customer instance credentials that all the subsequent installers will use. Within DomainManager, you should also allocate Windows user accounts to the Configuration, Setup, and WebView (if required) groups.
  • Step 3. Run ICMDBA.exe on Logger A. Use ICMDBA to create the Logger Data and Log databases. These need to be sized correctly based on the required retention periods set in the purge cycles and the expected contact center size (agents, skill groups, and call volumes).
  • Step 4. Connect to the Web Setup page and configure Logger A. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.
  • Step 5. Start the Logger A service. Start the service using ICM Service Control, and observe that each process starts correctly. The csfs process will complain that it is "Waiting for Link Enable," and the histlogger will be waiting for Message Delivery Service (MDS) messages. This is normal and nothing to be concerned about.

Installing UCCE Router A

The steps for installing UCCE Router A are as follows:

  • Step 1. Run setup.exe and install the Router A node. The setup.exe installer performs a base installation of the UCCE application onto the server. Include a maintenance release if required.
  • Step 2. Connect to the Web Setup page and configure Router A. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.
  • Step 3. Start the Router A service. Start the service using ICM Service Control, and observe that each process starts correctly. The router will not go active because it cannot see any PGs. MDS will also complain that Synchronizer operation is suspended as it cannot see the Side B router.
Figure 6-10

Figure 6-10 Autoreboot Warning Message

Installing UCCE Peripheral Gateway 1A

The steps for installing UCCE peripheral gateway 1A are as follows:

  • Step 1. Browse to the Unified CM server to obtain JTAPI. Download the JTAPI client from the Unified CM server and run the installation process on the PG.
  • Step 2. Run setup.exe and install the PG1A node. The setup.exe installer performs the entire PG installation because web setup is not currently enabled for PGs.

    Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets. To configure the Peripheral Interface Manager (PIM), you need to specify physical and logical controller IDs. These are integer values that are defined only when a PG is created in PG Explorer. The PG Explorer tool is not available until the administration workstation is configured. Many engineers specify dummy values at this point and then later return to setup after they have created the PG in PG Explorer. However, because this is the first PG, you could enter the value 5000 for both the physical and logical controller IDs because these are the default values created by PG Explorer. But be aware: How you configure subsequent peripherals determines which values are used. It is recommended that you double-check these values after you have used PG Explorer!

  • Step 3. Run setup.exe and install the CG1A node. During the installation of the CTI Server, make a note of the client connection port number. This number will be required when configuring the CTI OS Server.
  • Step 4. Start the PG1A service. Start the service using ICM Service Control, and observe that each process starts correctly. Observe that the PG connects to the central controller (Side A only). Error messages might appear in the process windows to say that the PG is not defined.
  • Step 5. Run setup.exe and install the CTI OS 1 Server. This is a different setup.exe than the PG/CTI Server. As with the PG, for the CTI OS installation, you need to enter the peripheral ID of the Unified CM server. If you are not confident of what the ID is going to be, you should not perform this step until after you have installed the AW and created the PG with PG Explorer.

Installing UCCE AW HDS 1

The steps for installing UCCE AW HDS 1 are as follows:

  • Step 1. Run setup.exe and install the AW Distributor node. The setup.exe installer performs a base installation of the UCCE application onto the server. Include a maintenance release if required.
  • Step 2. Run ICMDBA.exe on AW HDS1. Use ICMDBA to create the HDS Data and Log databases. These need to be sized correctly based on the required retention periods set in the purge cycles and the expected contact center size. My preference during setup is to perform the full install of an AW and HDS; however, the minimum requirement is only for an AW so that the PG can be configured.
  • Step 3. Connect to the Web Setup page and configure the AW. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.
  • Step 4. Start the AW Distributor service. Start the service using ICM Service Control, and observe that each process starts correctly. Observe the UpdateAW process to see whether the Waiting for New Work prompt appears. At this point in the installation process, you can perform the basic configuration of the PGs. Before you configure a PG, however, you must also create a default agent desk setting because this is a required option for the Unified CM PG!
  • Step 5. Validate the system. After the PG has been configured, check the PG and router processes. If the PGUser username and password are correct, the PG should be connected to the Unified CM server and its PIM and JTAPI processes should be ACTIVE. The CTI and CTI OS server processes should also be ACTIVE. The router MDS process should be In Service.

Installing UCCE Logger B

The steps for installing UCCE Logger B are as follows:

  • Step 1. Run setup.exe and install the Logger B node. The setup.exe installer performs a base installation of the UCCE application onto the server. The installer checks for third-party applications such as SQL Server and prompts the engineer if they are missing or configured incorrectly. Include a maintenance release if required.
  • Step 2. Run ICMDBA.exe on Logger B. Use ICMDBA to create the Logger Data and Log databases. These need to be sized correctly based on the required retention periods set in the purge cycles and the expected contact center size (agents, skill groups, and call volumes).
  • Step 3. Connect to the Web Setup page and configure Logger B. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.
  • Step 4. Start the Logger B service. Start the service using ICM Service Control, and observe that each process starts correctly. The csfs process will complain that it is Waiting for Link Enable, and the histlogger will be waiting for MDS messages. This is normal and nothing to be concerned about.

Installing UCCE Router B

The steps for installing UCCE Router B are as follows:

  • Step 1. Run setup.exe and install the Router B node. The setup.exe installer performs a base installation of the UCCE application onto the server. Include a maintenance release if required.
  • Step 2. Connect to the Web Setup page and configure Router B. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.
  • Step 3. Start the Router B service. Start the service using ICM Service Control, and observe that each process starts correctly. Routers A and B should now perform a state transfer and start running in duplex operation.

Installing UCCE Peripheral Gateway 1B

The steps for installing UCCE peripheral gateway 1B are as follows:

  • Step 1. Browse to the Unified CM server to obtain JTAPI. Download the JTAPI client from the Unified CM server and run the installation process on the PG.
  • Step 2. Run setup.exe and install the PG1B node. The setup.exe installer performs the entire PG installation as web setup is not currently enabled for PGs. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.
  • Step 3. Run setup.exe and install the CG1B node. During installation of the CTI Server, make a note of the client connection port number. This number will be required when configuring the CTI OS Server; this number will also be different than the port number obtained for CTI Server A.
  • Step 4. Start the PG1B service. Start the service using ICM Service Control, and observe that each process starts correctly. Observe that the PG connects to the central controller.
  • Step 5. Run setup.exe and install the CTI OS 1 Server. This is a different setup.exe than the PG/CTI Server. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.

Installing UCCE AW HDS 2

The steps for installing UCCE AW HDS 2 are as follows:

  • Step 1. Run setup.exe and install the AW Distributor node. The setup.exe installer performs a base installation of the UCCE application onto the server. Include a maintenance release if required.
  • Step 2. Run ICMDBA.exe on AW HDS2. Use ICMDBA to create the HDS Data and Log databases. These need to be sized correctly based on the required retention periods set in the purge cycles and the expected contact center size.
  • Step 3. Connect to the Web Setup page and configure the AW. Work through the configuration pages, entering the correct values that have been previously documented in the installation spreadsheets.
  • Step 4. Start the AW Distributor service. Start the service using ICM Service Control, and observe that each process starts correctly. Observe the UpdateAW process to see whether the Waiting for New Work prompt appears.

After the final AW HDS has been installed, the remaining components can be installed in almost any order. I prefer to get the remaining PGs configured and then move on to installing the base software for the IVRs, followed by the Cisco Agent Desktop servers.

Implementation Testing

The primary goal of UCCE implementation testing is to uncover problems with the application or its configuration that have occurred during the initial installation. Testing an entire UCCE solution can be complex because of the amount of customization that takes place with the call routing and reporting. Fortunately, testing the core installation is more focused on ensuring that the platform has been installed correctly and provides the required fault tolerance during failure scenarios.

Software testing is a formal engineering process that should be completed by an engineer who did not perform the installation. During testing, the test engineer works through a test specification composed of multiple prioritized test cases. Each test case details a series of scenarios or parameters. The results of the test case, as defined in the list that follows, are documented in the test results document:

  • Pass: The test case was executed per the test specification, and the result matched the expected result.
  • Fail: The test case was executed per the test specification, and the result did not match the expected result.
  • Not run: The test case was not executed. The reason should be documented.
  • Descoped: The test case was not executed because the feature or function that it tests is unavailable and will not be available during the test period.

Prior to performing the testing, the test manager receives direction from the project manager and the end customer on what is an acceptable testing exit criteria. In an ideal world, you might expect a 100 percent pass rate; however, this is unusual for a large, complex software deployment. A typical customer might require that 100 percent of the priority 1 and 2 tests pass successfully and require at least an 80 percent pass rate for the priority 3 and 4 test cases. Other customers might request an overall 75 percent pass rate, with a documented action plan to resolve the remaining priority 1 and 2 issues before going live.

The test cases that will be created to test the core UCCE application installation comprise only a small subset of the overall UCCE solution test specification. Testing for the overall UCCE solution would need to cover all the routing, reporting, agent desktop, Unified CM, IVR, and third-party platforms. Comprehensive testing, even for a modestly sized deployment, can require a considerable effort and usually requires a dedicated testing team. However, performing a proper systems test is crucial to give the customer confidence that its solution is fit for purpose.

As previously mentioned, the focus for testing of the core UCCE application installation is to ensure that the software components have been configured correctly and that the nodes/processes fail over correctly should a problem occur. The areas under test can be summarized as follows:

  • Configuration checking
  • Router process redundancy
  • Logger synchronization
  • AW synchronization
  • Peripheral connectivity
  • PG process redundancy
  • CTI Server process redundancy
  • CTI OS Server process redundancy
  • Router public network failure
  • Router private network failure
  • PG public network failure
  • PG private network failure
  • Individual server failures

Table 6-4 gives an example test case for testing PG redundancy.

Table 6-4. Example Test Case for Testing PG Redundancy

Test Case ID

UCCE-CORE-PGR001

Test Engineer

B. Smith

Date

21st March 2011

Test Result

Pass

Test Category

UCCE Core Installation

Test Title

Failover of Active Unified CM PM to Idle PG

Test Purpose

To ensure that the duplex PG pair are in communication so that should the active PG fail, the idle PG will resume control

Prerequisites

IP communication exists between both PGs and the Unified CM subscribers.

  • The chosen PG's PIM and JTAPIGW processes are both active. (the assumption is that the active PG is PG1A; if not, adjust the test procedure accordingly.)
  • The failover destination PG has its PG processes started and is currently in an idle state.

Procedure

Establish a remote control session to both PG1A and PG1B.

  • Using ICM Service Control, select the Cisco ICM cus01 PG1A service and select Stop (or Cycle).
  • The PG1A processes begin a graceful shutdown.
  • PG1A notifies PG1B that it is shutting down.
  • Observe that the PG1B PIM and JTAPIGW processes activate.
  • PG1B should now be active and be in communication with the Unified CM.

Notes

Summary

In this chapter, you worked through many of the important steps that should be covered before and during the installation of the core UCCE components. In particular, the key learning points from this chapter can be summarized as follows:

  • Cisco recommends a lifecycle approach to platform deployment.
  • Several design tools are available to ensure that the solution is designed and sized correctly to meet the needs of the enterprise.
  • UCCE solutions should be deployed only by a Cisco-approved ATP partner.