NETSCOUT Security Operations Center – Tales from the Trenches

NETSCOUT Security Operations Center – Tales from the Trenches

By Gareth Tomlinson, director, Security Operations Center, NETSCOUT, and Carlos Morales, vice president and general manager, DDoS Mitigation Services, NETSCOUT

The Arbor Cloud service provides the best possible protection from DDoS via a trifecta of defense: The world’s largest purpose-built DDoS network, powered by the leading technology in DDoS protection and run by the Arbor Cloud Security Operations Center (SOC), an elite team of DDoS experts. In our experience, DDoS protection is something of a team sport, and you’ll get the best protection by baking expertise, collaboration, and process into the service, both initially and on an ongoing basis.

Here are some of the attacks in the past six months that drive home the importance of the team and the network behind the technology:

  1. Performing DDoS surgery on the Fly

One attack mitigated by the SOC combined TCP SYN flooding with reflection/amplification and carpet-bombing techniques directed against a network operator in the Middle East. Significant in size and scope, the attack commenced with no warning, a circumstance that is more common to a subscriber-oriented attack than attacks on the company as a whole. The botnet was unique in that it was largely sourced from Amazon, AWS, and Google address spaces, so it was hard to filter without dropping legitimate traffic. The attack also targeted some areas of the network operator’s network that had not been previously provisioned in Arbor Cloud. The SOC did an emergency provisioning of the missing subnets to get the entire network under protection quickly. Because of the complexity of the attack, the team had to perform packet-level analysis of the traffic to pinpoint and block a combination of distinct attributes that characterized the illegitimate traffic. The SOC’s DDoS experts were uniquely capable of reacting quickly and performing real-time analysis to get results for the customer.

  • Lesson learned: Having the right technology is not always sufficient—you must also stay up to date on operational practices. For example, make sure that you expand your DDoS protection service to match network expansions. It can lead to a delay in getting coverage during attacks.
  1. Test Against the Best

Even the most well-prepared companies need to maintain their vigilance. A very well-protected, multinational, financial services firm with great security practices sustained an extremely complex attack that caused some data loss for about ten minutes before the attack was mitigated. The attackers used a new DNS mechanism whereby they attacked with a combination of random nonsense DNS host names combined with plausible but incorrect host names. With hundreds of thousands of domains hosted by the firm, it is challenging to discern legitimate user queries from false queries. It was again necessary to analyze the traffic and figure out the attack based on patterns and commonality. 

The SOC team did just that and were able to get a quick handle on a very difficult attack. Surprisingly, the attack went away after just 30 minutes with no follow up seen to date. Due to the complexity and relative brevity of the attack, the SOC believes that the attack was somebody testing out new botnet capabilities against a known well-equipped and well-protected target.

  1. Defenses Should Do No Harm

An Eastern European financial services firm experienced a set of somewhat complex TCP SYN-ACK reflection attacks that carpet-bombed the firm’s entire internet IP address space. The Arbor Cloud auto defenses quickly went into effect, and all of the customer’s services continued to operate normally. However, internal users from the company found that they were not able to use the internet. It turns out that the attack was impacting the company’s internet proxies, causing the company’s employees to lose access to the internet. A review of the situation revealed that the same mitigation templates had been applied to internet proxies that were in place for internet-facing services, because the focus had always been on protecting the services. Defense techniques need to be different when protecting outbound proxy traffic versus inbound server traffic. The issue was remedied in short order and traffic was restored to the company’s employees. The attack, which lasted a day and a half, caused a short impact to the company’s outbound internet traffic—but in the absence of proper DDoS protection, the organization could well have been off the internet for the entire time.

  • Lesson Learned: A successful defense has to take into account all aspects of a network as part of the preparation process.
  1. School’s Out

A well-protected educational network with a good deal of on-premises protection was attacked by another university, with enough effect that the impacted school had to send its traffic entirely to Arbor Cloud for mitigation. Just one university was able to generate enough attack traffic to overwhelm the entire infrastructure of the second university. This shows the scale that DDoS attack capabilities have achieved and the importance of protecting your network from partners and peers, in addition to the rest of the internet. Both institutions think that the attack was student-led. This is eerily reminiscent of some of the earliest DDoS attacks ever, in which university students tried to knock each other off-line on the ARPANET. 

Download the latest Threat Intelligence report for the latest research from NETSCOUT’s ATLAS Security Engineering and Response Team (ASERT).