The (Cyber Threat) Intelligence cycle
The Intelligence Cycle is the process of developing raw information into finished intelligence for policymakers to use in decision-making and action.
from The Federation of American Scientists (FAS)
Over the years, and all around the world, intelligence agencies have developed well-structured processes for collecting information, analyzing it, and turning it into actionable and strategic intelligence - the intelligence cycle is one of these methodologies.
What is the intelligence cycle, and how can it be applied to the cyber realm and thus become “The Cyber Threat Intelligence Cycle”? In this post, I will try to answer those questions.
The intelligence cycle has five steps:
Planning & Direction
Analysis & Production
Distribution & feedback
Let’s dive into each of them:
Planning & Direction
The planning and direction phase is when the decision-makers weigh in on different aspects of the business and decide what the focus of the intelligence “community” should be. In classical intelligence, the intelligence community can be the actual agencies such as NSA, CIA, etc. In Cyber Threat Intelligence (from now on, we’ll use CTI for short), we refer to the community as the organizational threat intel team, the malware researchers, network and SOC analysts, and others.
In classical intelligence collection, policymakers like military and government officials set the direction based on national interests. For example, there might be targets information about which will benefit defense or economic interests. The direction chosen would also include a clear definition of whether the collection should be for strategic or operational goals.
For example, in 2018, a strategic goal might be “How the events in the middle east are going to affect oil prices in the next 5–10 years” while a tactical one might be “What are the ground troop movements of the forces in Syria.”
Similarly, in CTI, the direction will be decided by the heads of the security programs in the organization - the CISO, CSO, or maybe even the SOC manager. Decisions made would base on an analysis provided to the decision-making team, as well as risks, business plans, and the allocated budget CTI in the organization. In classical intelligence, in contrast to CTI, budget constraints are usually evaluated at later stages or perhaps even ignored (since some intel is so crucial that extra budgets will be assigned).
Examples of questions on the strategic side:
“What types of adversaries target the oil and gas industry?”
“What types of Infosec programs and systems should we invest in?”
On the functional/tactical side:
“What IOCs (Indicators of compromise) can we find and block in our network to defend from the Mirai Botnet?”
Once the direction is set, the collection begins.
In classical Intelligence gathering, you have all the well-established disciplines of intelligence gathering
SIGINT & ELINT - Signals intelligence and electronic signal gathering and analysis (for example, monitoring submarines using SONAR) and communication signal gathering, which includes interception of calls, faxes, emails, etc.
HUMINT - Human intelligence, which includes human espionage (spies), diplomatic attachés, interrogation of prisoners, etc.
VISINT - Visual intelligence, which includes satellites, drones, and other methods of capturing images like long-ranged cameras.
OSINT - Open Source intelligence, which includes reading newspapers, watching TV, listening to Radio, and browsing the web for news about our targets.
When we are talking about CTI - these fields change a bit, and although we use multiple techniques based on the same logic, we adapt them to the cyber realm.
In this case - SIGINT becomes ingesting and collecting logs, feeds, honeypot data, and other sources of machine-generated data. VISINT becomes the extraction of texts from images and HUMINT the use of avatars to infiltrate and communicate with malicious actors on underground forums and the Dark web, or just collection of data from social networks.
In classical intelligence - most data is gathered from covert channels; To the contrast, in CTI, most (not all) of the data is collected using OSINT (open-source intelligence) resources such as feeds, malware labs, honeypots, social network scrapping, and analysis, reports published by security/IT vendors and others. But OSINT (even dark web OSINT) will only get you so far in your intelligence-gathering practice. Therefore you can choose one of the other options (or both):
Switch to “offensive” or “active” gathering, such as grabbing data from botnets command and control servers, buy data from criminals, or even “hack the hacker” (which is borderline, sometimes entirely, illegal).
Turn to threat data sharing with other organizations to obtain more precise and targeted data. Data sharing does exist in classic intelligence, as you can see in this article on how an Israeli IDF intelligence unit 8200 stopped a terrorist attack in Australia - but it is not as frequent nor as publicized.
There are many sharing platforms for CTI - some are commercial, and some are free, some require clearance as government entities operate them (e.g., national CERTS) and others require vetting by other members of the community.
Once the data is collected, The next step will be processing it into a more usable format.
In both classical intelligence and CTI, we need to process vast amounts of data. Still, while traditional intelligence has multiple types of sources, data analyzed in CTI is mostly (not all of it, of course) machine-generated.
It is interesting to note that most breached organizations had information about the breach in their logs for months and sometimes years before the breach is discovered. Still, those organizations were unable to detect the breach as the relevant signals were “drowned” in a mass of poorly processed data — either by human analysts or by software solutions.
We will often see the following types of actions taken when processing CTI data:
Parsing of logs and feeds
Extraction of data from images and PDF files (sometimes other files formats like GIS data to track an actor’s location or analysis of virtual machine images to find traces of a hack might also be of interest)
Translation of the data into canonical representation - imagine the data stored for some FQDN “subdomain.domain.com” - a well-built canonical representation will only store general data for the parent domain (such as whois info) in the parent record and will point the subdomain records to it.
Post-processing and data enrichment - this could include things like extraction of FQDNs and resolutions of IP addresses from malicious URLs or collection of whois data for FQDN.
Translation - although less common in CTI than in classical intelligence, translation from language to language is still an issue when you need to translate information from message boards or documentation (for example, Russian and other eastern European languages which are common in the cyber-crime underworld).
Another issue that is a significant part of the processing stage is the addition of confidence scores to the information collected. It’s usually collected on several levels - confidence in the source, confidence in the type of data and in the data itself.
Analysis and production
Finally! We have collected the data, made it usable, and now it is time to use it to answer the questions that we were sent to research in the first stage of the intelligence cycle.
In this stage, we combine different data sources that have been gathered and processed in the previous steps and use those to pivot on data points, recognize patterns and correlate the information to create more detailed stories and increase confidence for events.
In traditional intelligence, unless at least two sources can collaborate a piece of data, it will not be used. In CTI, on the other hand, we will usually use low confidence data pieces for metadata enrichment, noting that this is not high confidence. While some organizations will choose to implement a policy that blocks low confidence IOCs due to high risk from a threat, it isn’t a common practice. (For example, think about an organization that doesn’t mind blocking specific sites or users with a high probability of false positives as it chooses to prevent any possibility of being attacked by ransomware).
While in traditional intelligence, tactical intelligence might be information about the location of a terrorist cell, or the time when they are planning to set off a bomb so that military and law enforcement could prevent it. In CTI, tactical information is usually an automatically generated feed of IOCs that is ingested by security solutions such as Firewalls, IDS/IPS, and AVs. The main difference, in this case, will be that in classical intelligence, the output will mainly be used by humans, while in CTI, tactical intelligence’s primary usage will be by machines and will not have humans in the loop.
Distribution & Feedback
Finished intelligence products take many forms depending on the needs of the decision-maker and reporting requirements. An intelligence organization or community typically establishes the level of urgency of various types of intelligence. For example, in traditional intelligence, an indication and warning (I&W) bulletin would require higher precedence than an annual report. The same holds in CTI where feeds that are auto-generated and auto-consumed by systems are, by definition, more urgent than an overview of some cyber gang tactics, techniques, and procedures (TTPs) or a cyber espionage campaign writeup.
The feedback loop is of the utmost importance in both classical intelligence and CTI. Its main goal is to adjust the requirements made by the decision-makers, that said, tactical intelligence can also benefit from feedback. In CTI, for example, collecting feedback about indicator matches can be used for false-positive mitigation and usefulness of data (what is the usefulness of thousands of indicators that are introduced to solutions but never match to real activity?)
To summarize, just as adding the words “Cyber Threat” to the terms “Intelligence cycle” seems pretty natural, so does the migration of traditional intelligence analysis procedures and techniques into the cyber realm.
More information about booking training courses and workshops for your organization are available on damsky.tech/training