The Apache Log4j remote code vulnerability discovered in early December has the entire cybersecurity industry—from practitioners to vendors—scrambling to understand the exploit, identify impacted systems, and determine the best response, especially if quickly updating systems isn’t possible.
Severity and impact
If you don’t already know what the Log4j 0-day exploit is and what it means, this article is a primer, but here are a few quick facts:
Vendor application: Apache Log4j (v2)
CVSS Score: 10.0 Critical
Impact: Complete across Confidentiality, Integrity and Availability
CWE-502: Deserialization of untrusted data
Description: Allows remote code execution (RCE) by logging a specific string
Log4j is being deemed the worst security vulnerability by the industry. Nation-states and crypto miners have already seized the opportunity for deep, profitable exploits. What earns this vulnerability such notorious accolades? The ease of exploitation, the depth of the impact, and the widespread use of the Apache log library. Log4j is used by most Java apps, embedded in countless other applications, and included in open-source projects like Apache Struts.
What does it take to go from potential target to compromised? One vulnerable server to accept and log the malicious string using Log4j. This can be done over any protocol, including web requests with the most common protocol. Including it in the user agent field often gets logged. Discovering where this exists in your environment is very difficult, and the potential is very high for compromise to external servers to spread internally.
There are tools available to help identify it in your organization, from vulnerability scanners to stand-alone testing tools like Huntress. Professional services can help organizations struggling to identify Log4j in their environments. Sirius’ professional services team offers deep experience to help you assess your level of risk and develop a remediation plan.
Act now
Our recommendation is to assume you’ve been breached. Immediately amplify your monitoring for internal attacks, and then move to mitigate the impact.
Step 1: Identify external custom and third-party applications that use the Log4j logging library. Validate patching and configurations and ensure that random internet addresses are blocked. This might require a vulnerability scan or manual verification. Egress filtering is a basic measure, but it works—blocking outgoing requests from external servers, especially on LDAP ports. Until you can complete this, use firewalls and web application firewalls (WAFs) to block and monitor any searches for Log4j. Keep in mind that WAF evasion and obfuscation methods are already in the wild.
Step 2: Identify and secure your internal apps. These are also at risk even without external access. Attackers can leverage other network access methods to seek, find and exploit internal instances. Because there are often multiple factors of internal apps in use, a manual process will be tedious and time-consuming.
Mitigation resources
Individual manufacturers are releasing patches and notices of impact, and it’s worth your time to investigate these. Major security vendors from Cisco to Palo Alto Networks are affected, and the list is changing rapidly. As the severity and reach of this vulnerability continue to expand and recommendations continue to develop, the linked resources at the bottom of the page can help you stay on top of the situation.
Best tools for Log4j protections
If you have these solutions in place, make sure they are optimized and functioning as intended. Any solution listed below that is not already in your toolkit should be a strong contender for immediate implementation:
Segmentation/firewalling: The concepts of least privilege networking and micro-segmentation may be the most effective methodologies for protecting against the Log4J attack and similar attacks. This attack requires the server to connect back out to the attacker, which in general application design should not be allowed. Basic egress filtering should allow clients to talk to the server and respond to that request, but should never allow the server to create a connection to a client. There are exceptions that ignore this filter, and many servers do reach out to talk to other services. But this should be mapped with a specific default deny, blocking anything outbound that is not allowed. Micro-segmentation elevates this to a server-group level by not allowing clients to talk to servers or internal systems except on authorized protocols. The attack scope is limited and creates more methods to detect unusual network behavior.
Endpoint protection and endpoint detection and response (EPP/EDR): When combined with workload-specific tools, EPP/EDR tools are very good at identifying and preventing system compromise. While they do not generally prevent the vulnerability from being used, they do see the resultant command and control and any other efforts at full compromise and takeover of other systems. A strong EDR tool also logs all file touches, network connections, and processes started—making forensic analysis of what occurred much faster. Advanced EPP tools using user behavior indicators are much more effective at detecting these attacks because behavior after the compromise tends to look very similar.
Network detection and response (NDR): Analyzing network communications to and from servers is critical in identifying a breach in your systems. NDR helps identify command and control and abnormal network behavior. Store network data for as long as possible— 30 days at a minimum, but 90 days is better, and 12 months is best. Long retention allows you to look back when new indicators of compromise (IOCs) come out to see if any systems communicated with the recently identified systems, protocols or payloads.
Monitoring/SIEM/UEBA: Detection plays an important role in solving this issue because preventing everything isn’t possible. Reducing the time to detect and respond helps to limit the effect of these attacks and improves understanding of the true scope of the attack. Behavior-based solutions that combine alerts for identified attacker behavior with alerts for abnormal application behavior are powerful resources. Application behavior generally doesn’t change. When an app starts to make outbound connections, communicate with different protocols, or talk to new systems and accounts, it is detected with behavior-based monitoring.
Vulnerability scans: Trying to find where Log4j exists in an environment is very difficult because it is embedded in so many systems. Traditional vulnerability scanning will help identify Log4j instances, affected commercial applications, and products affected. More difficult areas, such as custom applications with embedded Log4j and IoT devices with firmware not easily identified, may need more advanced vulnerability scanning. Vulnerability scanner capabilities continue to improve results, but some systems and applications will need individual custom testing.
Extended implications of Log4j
Most attacks leveraging this vulnerability to date have been opportunistic rather than targeted, but this will change in the coming days as more groups and individual hackers take advantage. The subsequent attacks, countermeasures, and recovery resulting from this vulnerability will extend well into 2022 as the variation in exploit activity continues to grow.
This vulnerability will be a favorite for pen testers for years to come. External clean-up may appear relatively quick as attackers clean up after themselves and patching and updating efforts escalate. But complete detection is difficult, so this will lurk in systems undetected for a long time.
A changing situation
There are good recommendations for remediation, detection, and prevention currently available, but this situation and the information available are changing daily. Security researchers have invalidated some earlier recommendations, and WAF evasion and obfuscation methodologies that bypass early prevention and detection practices have been seen. With the full might of the security industry now looking at this vulnerability, evolution will continue. Expect updates to fix other issues found by the security community in the Log4j application to continue. We will update the information here as we all learn more. Mitigating this vulnerability is a long-haul project, and security efforts need to continue as understanding improves.
Detect and mitigate faster
Sirius offers professional services to help you find and resolve vulnerabilities in your system. Our team uses advanced methods to scan code and review logs, and we provide vulnerability scanning, pen testing, patching assistance, and other security assessments. To learn more, reach out to your Sirius representative or contact us today. Our Security team is ready to partner with you to bring your organization through the impact of this vulnerability.
