Backdoor Detection in Network Equipment
A breakdown into the Dec 2015 vulnerability in Juniper ScreenOS by carlu
The recent backdoor discovered in Juniper ScreenOS is big news at the moment.
In this article we aren’t going to focus on the politics, or demand answers from Juniper, instead we are going to look at the issue from the perspective of the network defender (the persons who look after such kit and/or the persons who are tasked with the security monitoring of it). We will assume there are actually 2 potential issues to address:
- Backdoors allowing unauthorised remote access
- Poor VPN encryption
What is a backdoor?
In simplistic terms, a backdoor is a method where a threat actor gains access to a network by avoiding the security measures. Typically this can be done through a vulnerability within the network which is exploited to gain access.
Backdoors allowing unauthorised remote access
According to Juniper’s KnowledgeBase article:
“The first issue allows unauthorized remote administrative access to the device over SSH or telnet. Exploitation of this vulnerability can lead to complete compromise of the affected system.”
There are some authentication log events to look for specific to the recently discovered backdoor, which grants access to the “system” user on the device:
Upon exploitation of this vulnerability, the log file would contain an entry that ‘system’ had logged on followed by password authentication for a username.
Normal login by user username1
2015-12-17 09:00:00 system warn 00515 Admin user username1has logged on via SSH from …..
2015-12-17 09:00:00 system warn 00528 SSH: Password authentication successful for admin user ‘username1’ at host …
Compromised login by user username2
2015-12-17 09:00:00 system warn 00515 Admin user system has logged on via SSH from …..
2015-12-17 09:00:00 system warn 00528 SSH: Password authentication successful for admin user ‘username2’ at host …
Note that a skilled attacker would likely remove these entries from the local log file, thus effectively eliminating any reliable signature that the device had been compromised.
If the device logs centrally to a log management system or SIEM, this is where to look for these events. If the unauthorised user disables monitoring, this entry could well be the last event seen from this device. However, relying on logged events alone to detect unauthorised access is a bad idea for two reasons:
- The identification of this type of activity as being potentially malicious has only been flagged recently. This back-door has been in place for a number of years.
- There is no guarantee that a decent indicator of such a wide ranging issue would create a log entry at all – a decent backdoor would not!
So then, how to detect such a problem?
Firstly it’s important to widen the scope of the issue – unauthorised access isn’t a problem unique to Juniper or any particular vendor, this is a problem that can potentially exist in pretty much any application or device for many reasons including device and application misconfiguration, insecure or malicious code, compromised versions of drivers and firmware etc. So from the network defender’s perspective it’s important to have general visibility of the indicators of unauthorised access attempts, rather than monitoring for the root causes making the headlines.
The techniques described below are used by e2e-asure to monitor network activity related to a myriad of devices, including but not limited to routers, firewalls, laptop and desktop endpoints, mobile devices and printers.
For the sake of relevance, the examples below relate to the detection of unauthorised VPN activity.
The following prerequisites are necessary:
Full visibility of network traffic to and from the VPN device
Full packet capture is key here, using port mirroring/SPAN (Switched Port Analyzer) or a hardware network tap copying traffic to a dedicated system or infrastructure running capture and analysis tools; a packet capture Swiss army knife. If the VPN endpoint is in the Cloud (e.g. AWS or Azure) access to capture the network traffic directly will not be readily available but there are still things that can be done to achieve a similar effect. This will be covered in a future article.
The ability to create alerts and signatures from the data collected
In order to investigate and respond there needs to be a means or being alerted when certain conditions are met using rules based on traffic flow; IP addresses, ports, protocols, etc.
The ability to create traffic whitelists or blacklists to filter out expected and authorised connections to devices by administrators and trusted third-parties
Whitelists work best here with alerts generated for all exceptions to the “known and allowed” list. It’s important to understand baseline traffic to detect anomalies here.
Basic questions to ask include:
- What type of traffic normally comes into the device?
- What type of traffic normally goes out of the device?
- What are the expected sources and destinations for this traffic?
A normal baseline for a fixed or site-to-site VPN device might look like this:
There may also be ICMP and NAT-T traffic in the mix as well. If established sessions to the VPN device are detected from an IP address that does not belong to an expected source, it’s time to investigate. It’s potentially wider then that – if any traffic is detected that doesn’t match the baseline rules created by following the above exercise, it’s time to investigate.
At this point it’s worth mentioning that only established connections should be flagged here. Devices accessible from the internet are scanned constantly and there’s a great deal of noise generated from failed connection attempts. These should be filtered out to ensure legitimate, accepted connections aren’t missed.
It’s highly likely that a backdoor will utilise an existing administrative service such as Telnet, SSH or Web Console (HTTP/ HTTPS) but depending on the backdoor it could use any number of different ports to connect.
There are a number of ways of capturing these connections but it really depends on the technology you are using to interpret packet capture.
How e2e-assure achieves backdoor detection in with Cumulo:
…and here’s a sample alert generated by a connection via SSH session from an unauthorised source using the PuTTY client:
Detection of backdoors with free/open-source tools
The following open-source network monitoring tools can be utilised to achieve steps 2 and 3 detailed above:
Snort and Suricata – IDS
Snort and Suricata are open-source Intrusion Detection Systems designed to be highly customisable. They both use the same rulesets and provide functionality for users to create and distribute their own custom rules for events not covered by the standard set of rules produced by the Snort team and third-parties.
FoxIT, a cyber security consultancy based in the Netherlands, have made available an emergency set of Snort rules designed to detect the Juniper ScreenOS backdoor.
Bro – IDS and Network Monitoring Platform
Bro is an additional open-source Intrusion Detection System that can be used to detect anomalous network behaviour as well as utilising preconfigured event-based alerting.
It features a powerful analysis engine that uses its own scripting language called “Bro-Script”. This language can be used to create custom alerts.
ELSA – Enterprise Log Search and Archive
ELSA is a receiver, indexer and archiver of syslog information and supports complex queries. ELSA can be used to collect, index and query syslog events generated by the VPN hosts as well as logs from the aforementioned Snort/Suricata and Bro IDS systems detailed above.
Security Onion
Security Onion is a Linux distribution that contains precompiled Snort/Suricata, Bro, ELSA, the Host Intrusion Detection software OSSEC, as well as host of additional open source network monitoring and traffic analysis tools too numerous to cover in this article. Security Onion can be set up in standalone or distributed sensor modes and can be run as a LiveCD requiring no installation for when urgent network monitoring is required.
Example queries for detecting unauthorised VPN activity using Bro and ELSA
By using ELSA to parse Bro logs it is possible to construct a query to alert on unauthorised access to VPN endpoints using the very simple query in the example below:
class="BRO_CONN" dstip="[IP of VPN endpoint]" SF
This searches all Bro connection logs for established connections to the VPN endpoint
It’s also possible to filter out whitelisted connections from known sources using the minus-sign query syntax
class="BRO_CONN" dstip="[IP of VPN endpoint]" SF -srcip=[known source IP] -srcip=[known source IP] proto=tcp -dstport=443
This query searches the Bro connection logs to return results similar to the example above, but to ignore the two known source IPs in the logs as well as ignore all SSL traffic to the VPN endpoint.
Mitigating the Issue at the Network Design Phase
The best way to mitigate the risk of unauthorised VPN access via backdoor is to prevent undesired traffic from reaching the backdoor in the first place. The CESG ‘Walled Garden’ places an inline bridging/transparent firewall above your VPN devices. This enables filtering of traffic to/from the VPN device thereby avoiding the remote backdoor issue from being accessible to the attacker.
Conclusion
The recent Juniper backdoor discovery isn’t the first and won’t be the last time a vulnerability of this magnitude is discovered affecting organizations’ internet-facing software and infrastructure.
It’s important to implement secure network design and apply protective monitoring techniques to detect and mitigate the threat of unauthorised access in general, rather than waiting for the next vendor security patch – which can follow an undiscovered vulnerability by months or sometimes years.