Assuring End-User Experience Across Multi-Vendor UCaaS Services

Customer Profile

This company is recognized as a leading transporter of natural gas and oil, while also making significant progress in developing renewable energy generation resources.

The company had earlier gone “all in” on voice, video, and Cisco TelePresence technology adoption, and these investments served them well when executive leadership announced a work-from-home (WFH) policy to safeguard personnel during the COVID-19 pandemic. While leadership was committed to increasing the safety of essential personnel required to operate certain Contact Center and Control Center functions onsite, in alignment with their business continuity plan, many employees began a near-overnight transition to their home offices.

For years, the company’s IT Operations team had looked to NETSCOUT as a trusted business partner to deliver service assurance and visibility solutions across the business, with past projects assisting their efforts to migrate applications to Software-as-a-Service (SaaS) platforms.

The Challenge

Their employees’ ability to embrace Unified Communications & Collaboration (UC&C) services as an effective substitute for in-person work duties enabled the company to safely operate energy systems, advance work on critical strategic and tactical projects, and continue high-quality-service delivery to their customers.

As a result of this hybrid workforce transition, the company joined the ranks of many of today’s leading enterprises that use two Unified Communications as a Service (UCaaS) solutions – in this instance, their go-to collaboration platforms were Cisco Webex and Microsoft Teams. This two-vendor solution approach enabled WFH employees to use the UC&C technology best suited for certain business or project needs (e.g., Microsoft Teams for collaboration-related activities, Webex for conferences). For this company, these technologies became so conjoined that IT leadership wanted to integrate the delivery of Microsoft Teams and Webex services to further enhance employee productivity and performance quality.

Executive leadership and IT Operations decided to fund an extended proof-of-concept (POC) phase to evaluate whether this effort could be satisfactorily completed and rolled out to employees.

Given both the company’s market position and the technical nature of this particular POC, IT leadership was understandably guarded about these project activities. In related communications with their contracted Premium Support Services (PSS) Engineer and other trusted NETSCOUT resources familiar to them, however, IT leadership described several issues they knew needed resolution for the POC to advance. For example:

• The quality of voice and video transmissions seen in outgoing streams was good, according to UC Monitoring workflows seen in their nGeniusONE Service Assurance views and reports. However, incoming streams showed poor mean opinion scores (MOS) and network packet jitter delivery delays.
• Two-second latency on external calls made for problematic communications – while voice performance would eventually “sync up,” these persistent delays made for poor-quality communications.
• When operating in the integrated Microsoft/Cisco POC environment, MS Teams users experienced “choppy frames” during video exchanges, which were both distracting and harmful to the quality of collaborations being conducted. Even worse, dropped session occurrences stopped certain calls in their tracks.

For IT leadership, there were visibility gaps into each vendor’s cloud environments that made it difficult to use nGeniusONE to monitor and analyze the traffic coming into the Cisco Webex and Microsoft Teams services from the internet. These cloud service edges represented a high concentration of the company’s traffic during the POC and required visibility to improve mean-time-to-knowledge (MTTK) regarding the reported problems.

While this project was only in the POC stage, finger-pointing began across the IT Operations divide regarding the root cause of these issues. As a result, the IT leadership team soon engaged their Microsoft and Cisco technology contacts about their concerns.

There was a lot at stake for this IT leadership team – technology-savvy company executives interested in the POC’s progress were quickly getting discouraged and considered stopping the POC.

Solution in Action

In addressing these issues, IT leadership turned to a solution familiar to them – the nGeniusONE platform, whose real-time performance analytics snapshots were based on NETSCOUT smart data derived from real-time network packet traffic, as well as user experience metrics generated from nGeniusPULSE nPoint tests at remote locations (e.g., WFH offices and remote sites). Using their long-time NETSCOUT PSS Engineer helped provide IT leadership with assurance that vendor-agnostic nGeniusONE troubleshooting would pinpoint root cause, regardless of the vendor or UC technologies involved.

With the nGeniusONE and already-deployed InfiniStreamNG (ISNG) smart visibility appliances and nPoints, IT leadership and NETSCOUT collaborated to use Service Dashboard, Media Monitor, and Session Analysis views to determine:

• The company’s network was not the root cause of these issues, as these analytics showed that the quality of outgoing UC streams was at acceptable levels. This led IT Operations to deduce the issue instead existed on the vendor technology side.
• While voice and video experience could not be visualized at the Webex and Teams cloud ingress with existing NETSCOUT data sources, the IT Operations team was able to see poor-quality UC traffic coming into their enterprise service edge.

Troubleshooting then turned to the WFH environments (i.e., client edges) to determine whether the source of the voice and video troubles related to the employee’s Wi-Fi, variety of different ISPs being used, or another element in remote users’ set-ups. In this regard, the PSS Engineer developed nGeniusPULSE synthetic tests, using virtual nPoints loaded on employee laptops, that assessed remote end-user experience of WFH employees utilizing certain Web browsers for Microsoft Teams access. While this NETSCOUT troubleshooting did reveal and resolve some Microsoft Teams interoperability issues, the nature of these problems was determined by IT Operations as being unrelated to root cause.

In the end, nGeniusONE UC Monitoring analytics provided evidence that root cause was related to the Cisco TelePresence environment. Based on the company’s collaborations with this important technology vendor, Cisco’s ability to update related media codecs addressed this root cause issue. The IT leadership team shared the nGeniusONE evidence into root cause identification, which ultimately persuaded Cisco to move forward with this update in good faith.

Coincidentally, the company’s C-level executive team had experienced similar Cisco TelePresence issues, so identifying and resolving this problem in the context of the POC was a significant win for IT leadership.

The Results

Absent NETSCOUT’s extensive visibility solutions for monitoring and analysis capabilities at the WFH users’ client edges, at network edges (where ISPs and WAN connect to data centers), and service edges in data centers and cloud (where UC&C and application services originated), this POC would have likely failed due to the poor-quality performance the users were collectively experiencing.

However, leveraging their nGeniusPULSE nPoint synthetic test results from the WFH client edge perspective along with nGeniusONE and ISNGs at the data center for Cisco TelePresence service edge visibility, this IT organization achieved quick MTTK, detailed evidence to share with their Cisco collaborators, and was able to proceed with a successful integration in their multi-vendor environment.

Perhaps the biggest benefit was their ability to optimize and deliver quality communications services to their employees, with fewer frustrations, while still meeting business continuity objectives during a stressful time.