Translated from 日本語 • View original 
In fact, looking back at recent incidents, there are overwhelmingly more cases where "logs existed but were not used" or "anomalies were recorded but missed" than cases where "logs did not exist".
In other words, the problem is not the amount of logs, but how the log operation is designed and utilized.
In this article, starting from the question, "Why can't attacks be detected even though there are logs?", we will explain typical pitfalls in security operations and how to improve them based on the concept of defense-in-depth. Read this as a hint on how to move your logs from "just keeping" to "defending" them.
Background: Logs continue to grow, but the reality is undetectable
With the spread of cloud, SaaS, remote work, IoT, and more, the amount and type of logs handled by companies are increasing every year. OS logs, application logs, network logs, authentication logs, API access logs, and more information available is incomparably richer.
On the other hand, the attack began to proceed more "quietly".
Rather than immediately engaging in sabotage after the initial intrusion, it is common to spend weeks or months exploring the interior to escalate privileges and expand laterally. During this time, all of the attacker's actions may remain logged.
Why can't it still be detected?
Often, the following challenges are in the background:
- Collecting logs, but not defining what to see
- The criteria for normality and abnormality are ambiguous, and the alerts are in the form of a formality
- Reliance on manual monitoring, analysis not keeping up
- Logs are scattered and cross-sectional correlation analysis is not possible
These are more of an operational design issue than a lack of tools.
Attack methods and risks: Fear of missing the "signs" that appear in the logs
Modern attacks proceed not as a single event, but as a "successive stack of actions."
For example, the flow is as follows.
- Login from an IP address that is not usually used in the middle of the night
- Increased authentication failures in a short period of time
- Access to APIs and management features that are not normally used in business
- Change permissions and create new accounts
- Cross-sectional access to internal systems
These may look like "common operations" when viewed on their own. However, it is clearly unnatural to associate it in chronological order or context.
If you only look at the log at the point, you will miss this "line".
As a result, the discovery of a breach is the "final stage" of ransomware execution or information exfiltration, maximizing damage.
This means that the detection and judgment mechanism is not working more than a technical problem.
Countermeasures: A multi-layered approach to turning logs into "defenses"
Log-driven detection requires more than just a SIEM. It's all about design and operation.
1. Environment Preparation: Rethinking Log Design
First, organize "which logs are needed for what".
Rather than storing everything, it's important to prioritize them by purpose of detection, investigation, and trail. In addition, time synchronization and format unification of logs have a significant impact on the accuracy of analysis in subsequent processes.
2. Building a Monitoring Regime: Correlation Analysis and Automation
You need a system that can analyze logs across multiple sources, not just logs. Leverage SIEM and XDR to automatically detect combinations of behaviors that humans may not notice.
The key is to design to detect "abnormal chain of operations".
3. Test Operation: Validating Detectability
There is no point in making rules and ending it.
Based on pseudo-attacks and past incidents, we verify whether they can really detect them. If there are too many alerts, adjust the accuracy and bring it down to a level that can be handled in the field.
4. Production: Clarifying the response flow
Even if it is detected, it is meaningless if the response stops.
Define in advance the initial response, escalation, and containment decisions when an alert occurs, and create a state where anyone can see and act. If this is ambiguous, the detection will end with a "notification".
5. Continuous Improvement: Keeping Up with Threats and Environmental Changes
Update your log design and detection rules to accommodate new attack vectors, business system additions, cloud configuration changes, and more.
By incorporating cyber threat intelligence, you can prepare for unknown attacks.
Related articles:
What it should be: A state in which logs become "early warning" instead of "retrospective"
In an ideal security operation, logs serve as sensors to notice in the middle of an attack, rather than post-incident evidence.
To do this, the following must be achieved:
- Ensure that important actions are recorded
- The definition of anomalies is agreed upon by the organization
- Automatic detection and human judgment are divided into roles
- The process from detection to response is trained
This is also consistent with the Zero Trust and NIST CSF 2.0 mindset.
Related articles:
Conclusion
It is not uncommon for a situation where there is a log but cannot be detected.
But it's not that logs are useless, it's just that they're not designed to be used.
Logs, when properly designed, correlated, and continuously improved, can be a powerful defense.
Before turning to tool implementation, the first step to security operational maturity is to clarify what we want to detect and what logs we need to do so.
Attacks always leave a trace. Whether or not an organization can notice the traces of the damage will determine the size of the damage.
If you have issues such as "I have logs but I can't use them" or "I'm worried about optimizing and setting detection rules in my own environment," the shortest route is to consult with an expert. Colorkrew Security supports the design of practical log operations tailored to your environment.