{"id":31721,"date":"2019-10-31T21:42:43","date_gmt":"2019-10-31T18:42:43","guid":{"rendered":"https:\/\/prohoster.info\/blog\/threat-hunting-ili-kak-zashhititsya-ot-5-ugroz\/"},"modified":"2019-10-31T21:42:43","modified_gmt":"2019-10-31T18:42:43","slug":"threat-hunting-ili-kak-zashhititsya-ot-5-ugroz","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/threat-hunting-ili-kak-zashhititsya-ot-5-ugroz","title":{"rendered":"Threat Hunting: How to Protect Against 5% of Threats","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

95% of cybersecurity threats are known and can be defended against using traditional means like antivirus software, firewalls, IDS, and WAF. The remaining 5% of threats are unknown and the most dangerous. They account for 70% of the risk to a company because they are very difficult to detect and even harder to defend against. Examples include black swans<\/a><\/noindex> such as the WannaCry and NotPetya\/ExPetr ransomware outbreaks, cryptominers, and the Stuxnet \"cyber weapon\" (which affected Iran's nuclear facilities), along with many others (does anyone still remember Kido\/Conficker?) that are challenging to defend against using classic protection methods. We want to discuss how to combat these 5% of threats using Threat Hunting technology.<\/p>\n

\"Threat<\/a><\/noindex>
\n The continuous evolution of cyberattacks necessitates constant detection and response, ultimately leading us to the idea of an endless arms race between attackers and defenders. Classic defense systems can no longer provide an acceptable level of security where risk doesn't impact key business metrics (economic, political, reputation) without customization for specific infrastructures, but they do cover some risks. Even during the deployment and configuration process, modern protection systems find themselves in a catch-up role and must respond to the challenges of the new era.<\/p>\n

\"ThreatSource<\/noindex><\/p>\n

One response to the challenges of modernity for information security specialists may be Threat Hunting technology. The term Threat Hunting (hereafter referred to as TH) emerged a few years ago. The technology itself is quite interesting but still lacks any accepted standards and regulations. The diversity of information sources and the scarcity of Russian-language resources on this topic complicates matters. Therefore, we at 'LANIT-Integration' decided to write an overview of this technology. <\/p>\n

Relevance<\/h2>\n

\nThe TH technology relies on infrastructure monitoring processes. There are two main scenarios for internal monitoring \u2013 Alerting and Hunting.<\/a><\/noindex>Alerting (similar to MSSP services) is a traditional method that involves searching for previously developed signatures and indicators of attacks and responding to them. This scenario is successfully handled by traditional signature-based protection tools. Hunting (an MDR-type service) is a monitoring method that addresses the question, 'Where do signatures and rules come from?'. It is a process of creating correlation rules by analyzing hidden or previously unknown indicators and signs of an attack. Threat Hunting refers to this type of monitoring.<\/p>\n

\"Threat
\nOnly by combining both types of monitoring can we achieve protection that is close to ideal, yet there always remains a certain level of residual risk.<\/p>\n

\"ThreatProtection using two types of monitoring<\/i><\/p>\n

Here\u2019s why TH (and hunting in general!) will become increasingly relevant:<\/p>\n

\"ThreatThreats, protection measures, risks.<\/i> Source<\/a><\/noindex><\/p>\n

95% of all threats are already well-studied.<\/a><\/noindex>These include types such as spam, DDoS, viruses, rootkits, and other classic malware. Protection against these threats can be implemented with the same classical protection measures.<\/p>\n

During any project execution, 20% of the time is spent on 80% of the work,<\/a><\/noindex>while the remaining 20% of the work takes 80% of the time. Similarly, among the entire threat landscape, 5% of new-type threats will account for 70% of the company's risk. In a company where information security management processes are organized, we can manage 30% of the risk from known threats in one way or another, either by avoiding (eliminating wireless networks altogether), accepting (implementing necessary protection measures), or transferring (for example, to the integrator) that risk. However, protecting against zero-day vulnerabilities,<\/a><\/noindex>APT attacks, phishing, supply chain attacks,<\/a><\/noindex>cyber espionage, and national operations, as well as a large number of other attacks, is much more challenging. The consequences of these 5% of threats will be far more severe (the average loss for banks from the buhtrap group is 143 million<\/a><\/noindex>), than the consequences of spam or viruses, against which antivirus software protects.<\/p>\n

Everyone is likely to encounter 5% of threats. Recently, we had to install an open-source solution that uses an application from the PEAR (PHP Extension and Application Repository) repository. The attempt to install this application via pear install failed, as website<\/a><\/noindex> was unavailable (now there's already a placeholder on it), I had to install it from GitHub. And just recently, it was revealed that PEAR has fallen victim to supply chain attacks<\/a><\/noindex>.<\/p>\n

\"Threat<\/p>\n

We can also recall the attack using CCleaner<\/a><\/noindex>, the NePetya ransomware epidemic via the accounting software update module M.E.Doc<\/a><\/noindex>. Threats are becoming increasingly sophisticated, and a logical question arises - \"How do we resist these 5% of threats?\"<\/p>\n

Definition of Threat Hunting<\/h2>\n

\nSo, Threat Hunting is the proactive and iterative process of seeking and discovering advanced threats that cannot be detected by traditional protective means. Advanced threats include, for example, attacks such as APT, zero-day vulnerability attacks, Living off the Land, and so on.<\/p>\n

It can also be rephrased that TH is a process of hypothesis verification. This is predominantly a manual process with elements of automation, in which an analyst, relying on their knowledge and qualifications, sifts through large volumes of information in search of signs of compromise corresponding to an initially defined hypothesis about the presence of a specific threat. A distinctive feature of it is the variety of information sources.<\/p>\n

It should be noted that Threat Hunting is not a software or hardware product. It is not alerts that can be seen in some solution. It is not a process of searching for IOCs (indicators of compromise). And it is not some passive activity that goes on without the participation of cybersecurity analysts. Threat Hunting is primarily a process.<\/p>\n

Components of Threat Hunting<\/h2>\n

\n\"Threat
\nThe three main components of Threat Hunting: data, technology, people.<\/p>\n

Data (what?)<\/b>, including Big Data. All kinds of traffic streams, information about previously conducted APTs, analytics, data on user activity, network data, information from employees, information from the dark web, and much more.<\/p>\n

Technology (how?)<\/b> processing this data - all possible ways to process this data, including Machine Learning.<\/p>\n

People (who?)<\/b> Those who have extensive experience in analyzing various attacks, developed intuition, and the ability to detect an attack. Typically, these are information security analysts who must be able to generate hypotheses and find confirmation for them. They are the key link in the process.<\/p>\n

The PARIS Model<\/h2>\n

\nAdam Bateman describes<\/a><\/noindex> The PARIS model for the ideal TH process. The name hints at a famous landmark in France. This model can be viewed from two perspectives \u2013 top down and bottom up.<\/p>\n

In the threat hunting process, moving from the bottom up model, we will deal with numerous pieces of evidence of malicious activity. Each piece of evidence has a measure called confidence \u2013 a characteristic that reflects the weight of this evidence. There are 'concrete', direct proofs of malicious activity, allowing us to quickly reach the top of the pyramid and create an actual alert regarding a confirmed infection. Then there are indirect evidences, the sum of which may also lead us to the top of the pyramid. As always, there is much more indirect evidence than direct, meaning they need sorting and analysis, further research, and ideally this should be automated.<\/p>\n

\"ThreatThe PARIS Model.<\/i> Source<\/a><\/noindex><\/p>\n

The top part of the model (1 and 2) is based on automation technologies and various analytics, while the bottom part (3 and 4) relies on qualified people who manage the process. The model can be viewed from the top down, where in the upper blue section we have alerts from traditional protective measures (antivirus, EDR, firewall, signatures) that carry a high degree of confidence and trust, while below are indicators (IOC, URL, MD5, and others) that have a lower degree of confidence and require further investigation. The very bottom and thickest level (4) involves generating hypotheses, creating new scenarios for traditional protective measures. This level is not limited to the specified sources of hypotheses. The lower the level, the more qualifications are required from the analyst.<\/p>\n

It is very important for analysts not only to check a final set of predetermined hypotheses, but to constantly work on generating new hypotheses and options for their verification.<\/p>\n

Maturity model of TH usage<\/h2>\n

\nIn an ideal world, TH is a continuous process. However, since there is no perfect world, let's analyze the maturity model<\/a><\/noindex> and methods in the context of people, processes, and technologies used. Let's consider the model of the ideal spherical TH. There are 5 levels of utilizing this technology. We will examine them using the evolution of a specific team of analysts as an example.<\/p>\n

Maturity Levels<\/b>
\nPeople<\/b>
\nProcesses<\/b>
\nTechnologies<\/b><\/p>\n

Level 0<\/b>
\nSOC Analysts
\n24\/7
\nTraditional tools:<\/p>\n

Traditional
\nAlert set
\nPassive monitoring
\nIDS, AV, Sandboxing,<\/p>\n

Without TH<\/i>
\nWorking with alerts<\/p>\n

signature analysis tools, Threat Intelligence data.<\/p>\n

Level 1<\/b>
\nSOC Analysts
\nOne-time TH
\nEDR<\/p>\n

Experimental
\nBasic knowledge of forensics
\nIOC Search
\nPartial coverage of data from network devices<\/p>\n

Experiments with TH<\/i>
\nGood understanding of networks and application part<\/p>\n

Partial application<\/p>\n

Level 2<\/b>
\nTemporary involvement
\nSprints
\nEDR<\/p>\n

Periodic
\nIntermediate knowledge of forensics
\nA week per month
\nFull application<\/p>\n

Temporary TH<\/i>
\nExcellent understanding of networks and application part
\nRegular TH
\nFull automation of EDR data usage<\/p>\n

Partial use of advanced EDR capabilities<\/p>\n

Level 3<\/b>
\nDedicated TH team
\n24\/7
\nPartial ability to validate TH hypotheses<\/p>\n

Preventive
\nExcellent knowledge of forensics and malware
\nPreventive TH
\nFull use of advanced EDR capabilities<\/p>\n

Specific TH cases<\/i>
\nExcellent knowledge of the attacking side
\nSpecific TH cases
\nComplete data coverage from network devices<\/p>\n

Configuration tailored to needs<\/p>\n

Level 4<\/b>
\nDedicated TH team
\n24\/7
\nFull ability to validate TH hypotheses<\/p>\n

Leading
\nExcellent knowledge of forensics and malware
\nPreventive TH
\nLevel 3, plus:<\/p>\n

Using TH<\/i>
\nExcellent knowledge of the attacking side
\nVerification, automation, and validation of TH hypotheses
\nclose integration of data sources;<\/p>\n

Research capability<\/p>\n

development tailored to needs and custom use of API.<\/p>\n

TH maturity levels in the context of people, processes, and technologies<\/i><\/p>\n

Level 0:<\/b> traditional, without the use of TH. Regular analysts work with a standard set of alerts in passive monitoring mode using standard tools and technologies: IDS, AV, sandboxes, signature analysis tools.<\/p>\n

Level 1:<\/b> Experimental, using TH. Analysts with basic knowledge of forensics and a good understanding of networks and applied aspects can conduct one-time Threat Hunting by searching for indicators of compromise. The tools include EDR with partial data coverage from network devices. Tools are used partially.<\/p>\n

Level 2:<\/b> periodic, temporary TH. The same analysts who have already advanced their knowledge in forensics, networks, and applied aspects are tasked with regular engagement (sprint) in Threat Hunting, say, one week a month. The tools include comprehensive data research from network devices, automation of data analysis from EDR, and partial use of EDR's advanced capabilities.<\/p>\n

Level 3:<\/b> Preventive, frequent TH cases. Our analysts have organized into a dedicated team, acquiring excellent knowledge of forensics and malware, along with understanding the methods and tactics of the attacking side. The process is already being conducted in a 24\/7 mode. The team can partially verify TH hypotheses while fully utilizing the advanced capabilities of EDR with complete data coverage from network devices. Analysts are also capable of configuring tools to suit their needs.<\/p>\n

Level 4:<\/b> Leading, utilizing TH. The same team has gained the ability to conduct research, generate and automate the process of verifying TH hypotheses. Now, the tools have additional tight integration of data sources, software development tailored to needs, and unconventional use of APIs.<\/p>\n

Threat Hunting Techniques<\/h2>\n

\n\"ThreatBasic Threat Hunting Techniques<\/i><\/p>\n

To techniques<\/a><\/noindex> TH in order of the maturity of the technology used includes: basic search, statistical analysis, visualization techniques, simple aggregations, machine learning, and Bayesian methods. <\/p>\n

The simplest method is basic search, used to narrow down the research area through specific queries. Statistical analysis is applied, for example, to construct a typical user or network activity in the form of a statistical model. Visualization techniques are used for clear representation and simplification of data analysis through graphs and charts, where patterns in the sample are much easier to discern. The technique of simple aggregations on key fields is used to optimize search and analysis. The higher the maturity level achieved in the organization\u2019s Threat Hunting process, the more relevant the use of machine learning algorithms becomes. They are also widely used for spam filtering, malicious traffic detection, and fraud detection. A more advanced type of machine learning algorithms is Bayesian methods, which allow for classification, dimensionality reduction of samples, and topic modeling.<\/p>\n

Diamond Model and Threat Hunting Strategies<\/h2>\n

\nSergio Caltagirone, Andrew Pendergast, and Christopher Betz in their work \u201cThe Diamond Model of Intrusion Analysis<\/a><\/noindex>\u201d outlined the key components of any malicious activity and the basic relationships between them.<\/p>\n

\"ThreatDiamond Model for Malicious Activity<\/i><\/p>\n

According to this model, there are 4 Threat Hunting strategies that are based on the corresponding key components.<\/p>\n

1. Victim-focused strategy. We assume the victim has adversaries, and they will deliver \"opportunities\" via email. We look for enemy data in the email. Searching for links, attachments, etc. We seek confirmation of this hypothesis for a certain period (a month, two weeks), and if nothing is found, the hypothesis is invalid.<\/p>\n

2. Infrastructure-focused strategy. There are several methods for employing this strategy. Depending on access and visibility, some are easier than others. For example, we monitor domain name servers known for hosting malicious domains. Or we conduct a process of tracking all new domain registrations for known patterns used by the adversary.<\/p>\n

3. Capability-based strategy. In addition to the victim-focused strategy used by most network defenders, there is a capability-based strategy. This is the second most popular and focuses on detecting opportunities presented by the adversary, specifically 'malware' and the potential use of legitimate tools by the adversary, such as psexec, powershell, certutil, and others.<\/p>\n

4. Adversary-focused strategy. The adversary-focused approach concentrates on the adversary themselves. This includes the use of publicly available information from open sources (OSINT), gathering data about the adversary, their tactics, techniques, and procedures (TTP), analyzing past incidents, threat intelligence data, and so on.<\/p>\n

Information sources and hypotheses in Threat Hunting<\/h2>\n

\n\"ThreatSome information sources for Threat Hunting<\/i><\/p>\n

There can be many information sources. An ideal analyst should be able to extract information from everything around them. Typical sources in almost any infrastructure will include data from security tools: DLP, SIEM, IDS\/IPS, WAF\/FW, EDR. Additionally, various indicators of compromise, Threat Intelligence services, CERT data, and OSINT will also be standard information sources. Information from the darknet can also be leveraged (for instance, there may suddenly be an order for hacking the CEO's email, or a candidate for a network engineer position may have drawn attention through their activities), information received from HR (feedback about candidates from their previous job), and information from security services (such as the results of contractor checks).<\/p>\n

However, before utilizing all available sources, at least one hypothesis must be established.<\/p>\n

\"ThreatSource<\/a><\/noindex><\/p>\n

To test hypotheses, they must first be formulated. And to come up with many quality hypotheses, a systematic approach is necessary. The process of generating hypotheses is described in more detail in article<\/a><\/noindex>, this scheme is very convenient to use as the basis for the hypothesis formulation process.<\/p>\n

The primary source of hypotheses will be the ATT&CK matrix<\/b> (Adversarial Tactics, Techniques and Common Knowledge). It essentially serves as a knowledge base and a model for assessing the behavior of adversaries carrying out their activities at the latter stages of an attack, usually described using the concept of the Kill Chain. This means during the phases after an attacker has penetrated the internal network of an organization or a mobile device. Initially, the knowledge base included descriptions of 121 tactics and techniques used in attacks, each detailed in a Wiki format. Diverse Threat Intelligence analytics are well-suited as sources for generating hypotheses. Notably, the results of infrastructure analysis and penetration tests stand out \u2013 these are the most valuable data that can provide us with reliable hypotheses due to their foundation on specific infrastructure with its distinct vulnerabilities.<\/p>\n

Hypothesis Verification Process<\/h2>\n

\nSergey Soldatov presented a good diagram<\/a><\/noindex> with a detailed description of the process; it illustrates the hypothesis verification process in a particular system. I will outline the main stages with brief descriptions.<\/p>\n

\"ThreatSource<\/a><\/noindex> <\/p>\n

Stage 1: TI Farm<\/b><\/p>\n

At this stage, it is necessary to identify objects<\/b> (by analyzing them along with all threat data) and assign them labels indicating their characteristics. This can be a file, URL, MD5, process, utility, or event. By passing them through Threat Intelligence systems, labels need to be applied. For instance, this site was noted in CNC in a certain year, this MD5 was linked to a specific malware, this MD5 was downloaded from a site that distributed malware.<\/p>\n

Stage 2: Cases<\/b><\/p>\n

In the second stage, we examine the interactions between these objects and identify relationships among all of them. We obtain marked systems that are performing suspicious activities.<\/p>\n

Stage 3: Analyst<\/b><\/p>\n

In the third stage, the case is handed over to an experienced analyst who possesses extensive analysis expertise, and he delivers the verdict. He breaks down the code byte by byte to determine what, where, how, why, and the purpose behind it. This body was malicious, this computer was infected. He uncovers links between objects and verifies the results through the sandbox.<\/p>\n

The analyst's findings are passed on. Digital Forensics investigates images, Malware Analysis examines discovered artifacts, and the Incident Response team may deploy to the location to investigate something on-site. The outcome will be a confirmed hypothesis, identified attack, and counteraction strategies.<\/p>\n

\"ThreatSource<\/a><\/noindex>
\n \u00a0<\/p>\n

Summary<\/h2>\n

\nThreat Hunting is a relatively new technology capable of effectively countering customized, novel, and unconventional threats, which has significant potential given the increasing number of such threats and the complexity of corporate infrastructure. It requires three components: data, tools, and analysts. The benefits of Threat Hunting extend beyond preempting threats. It's important to remember that during the search process, we delve into our infrastructure and its weak points through the eyes of a security analyst and can further strengthen those areas.<\/p>\n

The first steps that, in our opinion, should be taken to initiate the Threat Hunting process within your organization.<\/p>\n

    \n
  1. Ensure the protection of endpoints and network infrastructure. Ensure visibility (NetFlow) and control (firewall, IDS, IPS, DLP) of all processes in your network. Know your network from the edge router to the very last host.<\/li>\n
  2. Study MITRE ATT&CK<\/a><\/noindex>.<\/li>\n
  3. Conduct regular penetration tests on at least the key external resources, analyze the results, identify the main targets for attacks, and close their vulnerabilities.<\/li>\n
  4. Implement an open-source Threat Intelligence system (e.g., MISP, Yeti) and conduct log analysis alongside it.<\/li>\n
  5. Implement an Incident Response Platform (IRP): R-Vision IRP, The Hive, a sandbox for analyzing suspicious files (FortiSandbox, Cuckoo).<\/li>\n
  6. Automate routine processes. Log analysis, incident creation, notifying staff \u2014 these are vast areas for automation.<\/li>\n
  7. Learn to interact effectively with engineers, developers, and technical support for collaborative work on incidents.<\/li>\n
  8. Document the entire process, key points, and results achieved, so you can refer back to them later or share this information with colleagues;<\/li>\n
  9. Remember the social aspect: stay informed about what is happening with your employees, who you hire, and who you grant access to the organization's information resources.<\/li>\n
  10. Stay informed about trends in new threats and ways to protect against them, enhance your level of technical literacy (including in the operation of IT services and subsystems), attend conferences, and communicate with colleagues.<\/li>\n<\/ol>\n

    \nI am ready to discuss the organization of the TH process in the comments.<\/p>\n

    Or come work with us!<\/b><\/p>\n