Why Targeted Decryption Increases Network Visibility

By Erik Hjelmstad

As the internet has grown and expanded from a simple way for a few scientists to communicate and exchange data to a ubiquitous communication and commerce tool for billions of people, the need for private and secure communications has grown as well. That’s why internet traffic is routinely encrypted. Even on internal networks, communications are often encrypted to secure data against a potential compromise. How can you use targeted decryption to regain visibility into such traffic?

Encryption and visibility

Application monitoring appliances are designed to let the end user troubleshoot problems on the network. These tools let the user see all of the different transaction types, when communications are first established with clients, and the sequence of commands that are sent between the client and the server. A lot of information can be obtained with products like nGeniusONE, but the appliances are not able to provide as much visibility into encrypted data as they are with decrypted data. HTTP compared with HTTPS is a great example. With HTTP, an application monitor can report on all of the GET, PUT, POST, and DELETE requests that are sent to the server, as well as the server’s response to these requests.

Another situation where it is important to decrypt encrypted traffic for inspection is to check for malicious activity. Many security appliances can detect reverse shells, injection code, and command-and-control (C2) bots. This traffic can only be identified and analyzed if the traffic is unencrypted. Most IDS and IPS appliances and other tools that are designed to detect malicious traffic are much less useful, if not useless, when the traffic is encrypted. This makes it essential to decrypt traffic that is being sent to and from unknown internet servers. For example, some vendors will use the Sequence of Packet Lengths and Times (SPLT), combined with information about which protocols and cipher suites are being used, to infer that malware is being used. To be clear, this is a valid approach for detecting many types of malware, but these methods simply cannot detect many other attack-chain behaviors. 

When to decrypt

Let’s look at the types of traffic typically decrypted, and why:

HTTPS

When people think of decrypting network traffic, most first think of web traffic. This is an important type of traffic to decrypt as many attacks propagate over HTTP and HTTPS connections. This traffic can be easily decrypted with an inline device. Once the traffic is decrypted, it’s important to ensure the traffic is sent only to those monitoring or compliance tools that need to see it.

SSH

Secure Shell (SSH) traffic is normally used by administrators to execute commands on remote systems. The benefit of encrypting this traffic should be obvious: passwords are not sent in plain text as they are in a Telnet or FTP session. But attackers know how to use SSH as well. As far back as 2003, and probably before, attackers used SSH tunnels to send files and hide outbound communication from a compromised host back to their network. This type of exfiltration could be stopped by an outbound firewall rule on port 22, but what if the attacker used a different port for this communication? By intercepting and decrypting SSH traffic on any port, this type of attack could be stopped even if an attacker has gained a foothold inside your network.

SIPS (Secure SIP)

Session Initiation Protocol (SIP) is a call control protocol used for Voice over IP (VoIP) networks. Secure SIP introduces the mechanism to protect this traffic via TLS encryption, which makes it more difficult to monitor. By decrypting this traffic and sending it to a monitoring tool, administrators can monitor their VoIP call quality. This is something that just cannot be done with encrypted traffic.

Ultimately, you should decrypt network communications to confidently detect and respond to many common threats. True network traffic analysis for the enterprise requires the ability to decrypt approved traffic for deep application-level inspection. Ideally, your decryption function should be dedicated. Although many products, such as next-generation firewalls, are capable of decryption, they fail to decrypt nearly as effectively or efficiently as a dedicated decryption product. 

Learn more in our webinar, “Monitoring Encrypted Traffic to Ensure Efficiency and Availability”

Watch the webinar

Erik Hjelmstad is senior technical marketing engineer at NETSCOUT.