IVR Outage Restoration System

There are two things right off the bat. First, this will NOT be worded like a press release. That is for the overview, the point from which I will assume you linked. Second, I will assume a basic knowledge of IVR technology as well as PBX and ACD as related to a Nortel Meridian 1 switch. OK? Lets go. Oh wait, I am doing my best to describe the operation of this system so I just know this is going to be long winded.

The IVR is a Periphonics Corporation VAS (SP). The SP consists of two sub-systems. There is proprietary hardware that handles the telephone and host interfaces. This is a S-100 bus system that is controlled by a Motorola 68030 microprocessor. The actual IVR applications run on a SUN Sparc Classic which sends and receives control commands via a local Ethernet LAN segment running TCP/IP. Application development is done using a Periphonics CASE tool called PeriProducer. This runs on a SUN Sparc 5 that is also connected to the local IVR LAN segment.

The PBX is a Nortel Meridian 1 running option 81 software. The IVR connects to the PBX as a 20 line ACD supporting standard 2500 series analog phone sets. This allows the IVR to "mirror" the Outage ACD, which is also 20 lines of digital phone sets. This mirroring allows calls to be switched from one ACDDN to the other ACDDN, but more on that later.

Of course there are problems associated with connecting to the PBX as an ACD agent. The major one being the fact the agent position can be logged out by a variety of circumstances; one being an INI of the switch. Normally a human agent would recognize this fact by looking at their digital phone set. Or, if they went off hook, they would hear precise dial tone (i.e. 330/440 hz) and know that their station was not logged in. They could then dial the access code to log their station in and be in business.

IVR, on the other hand, has an analog appearance (the digital appearance is proprietary to Nortel and not available to IVR vendors). The system just sits there, waiting to take phone calls. If agent positions are logged out, that particular port just doesn't take any calls. Because this system is only activated during an outage, this logged off condition could go unnoticed until it was too late.

To solve this problem, and at the same time retrieve additional information we established a connection between an SDI port on the PBX and an async port on IVR. The SDI port was configured to provide 30 second updates on the IVR ACDDN as well as the Customer Service Representative (CSR) ACDDN. This provides number of agents logged in, number of calls in queue and average speed of answer (ASA) for each DN.

If IVR ever sees that less than 20 agents are logged into its DN, it sets a flag in shared memory that tells each line to check its log status. If all 20 lines are not recovered (i.e. maybe a PBX card went bad, or was taken out of service) the new number of agents logged in becomes the standard. This prevents the IVR from flooding the PBX with log requests. Every 30 minutes IVR attempts to recover all 20 agents, so even if a card is taken out of service, and then replaced, IVR will recover automatically within 30 minutes.

Normally calls to our 863-9000 number go to an auto attendant (AA) function provided by the PBX. It simply tells them to push one for electric, two for gas and water or three for billing information. In the event of an electric outage, the IVR takes on this AA function. To accomplish this a "virtual" DN is placed before the AA. This is called a Control DN (CDN) and is a software controllable pointer for inbound calls. Normally it points to the AA, but in the case of an outage, commands are sent that change the pointer to the IVR ACDDN. After the outage commands are then sent to "point" the inbound calls back to the AA. Where do these commands come from?

The IVR of course. There is a second async connection between the IVR and the PBX. This connection allows the IVR to issue $l commands to the switch. This connection is also used to decide the current status of IVR, i.e. whether it is on or off.

Now the next logical question is, How does IVR decide when to issue these commands? Well that is going to change very shortly, but I'll tell you how it's done now. There is an application running on a dedicated phone line that is in charge of activating and deactivating the outage application. When a caller dials the number, IVR goes out on the administrative port and queries the switch on the current status of the CDN, which of course directly correlates with the status of IVR. The caller is then informed of the status and asked to enter their ID number. If they TT the correct number in IVR tells them that they can change the status (on or off) of IVR by pressing 1. The CDN is changed appropriately, the CDN status is queried again to confirm the change, and the results of the changed are voiced to the caller. This process allows ANYONE to check the status of IVR, but only selected people to actually change the status.

When the status of IVR is changed, a second program starts. This program has two out dial lists, one for pagers and one for verbal messages. IVR calls each pager and sends 9999999 to indicate activation and 0000000 to indicate deactivation. The verbal messages just basically say that IVR has been activated or deactivated.

OK. IVR has been activated and is ready to take calls. It answers each call with the message that Springfield is experiencing an electric outage, and then proceeds with normal AA functions. Callers that press two or three are routed to the proper phone number. Callers that press 1 go to the Outage Restoration System. If the caller does nothing (i.e. a rotary customer or someone that just doesn't want to play) they go to continuous speech recognition. Here they are asked to SAY one, two or three. Once again, two or three go their separate way. A one, or no response goes on to Electric Outage.

The call is now at Electric Outage. Even though some of the callers SAID one, or nothing at all we are not ready to label them as a DTMF vs. a CWR call just yet. Our surveys have shown that a large percentage of the callers that don't respond with touch tones are not really rotary customers at all. Some don't want to talk to a machine, while others feel they will get better service by talking to a live human being. Many times this is just not the case. Late at night, or during a large outage callers can wait on hold for 10 or 15 minutes, waiting for their faster service. If they elect to use touch tones we can log their outage and have them off the phone in less than a minute.

To try and convince callers to use the system, if queue times are greater than one minute, we announce them to the caller. We get the que time as the average speed of answer which is one of the stats from the async port on the PBX and is updated every 30 seconds. We then tell the caller that for faster service they may log their call electronically by using their touch tone telephone. If we do not detect touch tones the caller is sent to continuous voice recognition. Here they are asked to SAY their phone number. If they STILL don't respond we give up and put them in the live agent queue. If they do say their phone number, the call is marked as a CWR call and handled appropriately for the rest of the call.

We do a data base lookup on the phone number and if it is not found, the caller is referred to an agent. If the number is found, a screen is returned with all customer information filled in. We grab the numeric portion of the address (the "house number") and voice it back to the caller for confirmation. This both insures that we dispatch crews to the proper address as well as instill confidence in our customers mind. The ticket is then submitted.

After submission two things can happen. First, it can be a duplicate ticket. This means that the caller has already logged one ticket to that address and either doesn't have faith that the system actually logged it, or possibly they are looking for an update. We tell them that a ticket has already been logged and give them an option to talk to an agent. If they choose to transfer, we give them a revised waiting time in queue and transfer the call.

If it is a new ticket, the caller is informed that their outage has been logged and they are given a chance to talk to an agent. Once again, if they choose to transfer, we give a revised waiting time and transfer the call.

Some customers have a special flag that indicates there may be life support equipment at their location. If they log a ticket, we inform them not only that their ticket was logged, but also that we are aware of their special needs.

Statistics are gathered throughout the program. 48 different points monitor movement through the finite state machines as well as specific activities that happen in each state. This gives valuable data that is analyzed after each large outage. This data is then used to fine tune the application, its resources and make any recommendations for upgrade or expansion.

If you have any questions please e-mail me and I will do my best to answer them for you.

Virtually,
Todd Christell