Explain the importance of threat data and intelligence.
Given a scenario, utilize threat intelligence to support organizational security.
Intelligence-driven defense is a widely accepted approach to information security assurance and critical to the tasks you will perform as a cybersecurity analyst. In this topic you will discover the life-cycle approach to intelligence gathering and usage, and identify reliable sources of threat data.
SECURITY INTELLIGENCE AND THREAT INTELLIGENCE
Security intelligence is the process through which data generated in the ongoing use of information systems is collected, processed, integrated, evaluated, analyzed, and interpreted to provide insights into the security status of those systems. While most security intelligence gathering efforts focus on information about your systems (firewall logs, intrusion detection alerts, and so on), threat intelligence, or more specifically cyber threat intelligence (CTI), provides data about the external threat landscape, such as active hacker groups, malware outbreaks, zero-day exploits, and so on. CTI is typically produced in one of two formats:
- Narrative reports—Analysis of certain adversary groups or a malware sample provided as a written document. These provide valuable information and knowledge, but in a format that must be assimilated manually by analysts. This is most useful at providing strategic intelligence to influence security control selection and configuration.
- Data feeds—Lists of known bad indicators, such as domain names or IP addresses associated with spam or distributed denial of service (DDoS) attacks, or hashes of exploit code. This provides tactical or operational intelligence that can be used within an automated system to inform real-time decisions and analysis as part of incident response or digital forensics.
The combination of security intelligence and CTI data can be processed, correlated, and analyzed to provide actionable insights that will assist you in identifying security problems. For example, security intelligence reveals that DDoS attacks were perpetrated against your web services from a range of IP addresses by collecting log and network traffic data. Threat intelligence associates those IP addresses with a hacktivist group. By linking the two sources of intelligence, you can identify goals and tactics associated with that group and use controls to mitigate further attacks.
SECURITY INTELLIGENCE CYCLE—REQUIREMENTS AND COLLECTION
Security intelligence is about more than just data collection, although collection is a big part of the process. Information regarding potential security problems is hidden within massive amounts of raw data produced as a byproduct through the ongoing use of your information systems. The security intelligence cycle involves various steps you perform to not only collect data, but also to process and analyze it so you can obtain actionable insights, which are formatted and organized to provide decision makers with relevant and useful information.
The requirements phase sets out the goals for the intelligence gathering effort. This phase is also widely referred to as Planning and Direction. This phase should show how intelligence will support business goals, such as ensuring a trustworthy repository of company data. The analyst effort needs to be properly costed and resourced, with sufficient staffing levels and tools.
Goals can also be specified in more detail by creating use cases for each intelligence gathering and analysis activity. For example, an automobile manufacturer is highly exposed to compromises of electronics incorporated within its vehicles. Consequently, a vital use case will be implemented to investigate supply chain threats. By defining use cases, you can define specific requirements for intelligence sources and direct analyst effort to providing measurable results. There is a wide variety of potential data sources, some of which you may already capture, such as system and application logs. In other cases, you may need to enable additional logging or tracking capabilities in advance to ensure you have the data you need. Because the collection of some data requires advance planning and preparation, it is important to perform the planning step carefully and think through your intelligence requirements in advance. In a large organization, this should be conducted as a unified effort across departments and functional groups to ensure that the right data is being collected.
This phase should also consider any special factors and constraints that will determine requirements. For example, there may be regulatory or legal stipulations for types of data to gather and for how long it must be retained. There may also be technical constraints and challenges around import requirements for specific software tools.
Collection and Processing
Collection is usually implemented by software suites, such as security information and event management (SIEM). This software must be configured with connectors or agents that can retrieve data from sources such as firewalls, routers, IDS sensors, and servers.
** SIEM is usually pronounced "sim," though some prefer "see-em" or other variants (twitter.com/anton_chuvakin/status/922607118175256577?lang=en). **
As part of the collection phase, or as a separate phase, the data retrieved from different systems must be processed. Processing puts data into a consistent format so that analysis tools can operate on it effectively. For example, the source IP address might be recorded in many different positions or columns in various log sources. Processing ensures that this data point is referenced consistently, can be searched/indexed, and can be correlated across multiple sources. Some solutions may require extensive scripting or may involve extensive manual processing.
Another consideration for the collection and processing phase is to keep security data secure. Many of the logs used in security intelligence collection contain information that is not only useful to those protecting the organization's information systems but would also be useful to an attacker.
SECURITY INTELLIGENCE CYCLE—ANALYSIS, DISSEMINATION, AND FEEDBACK
The requirements and collection/processing phases establish a normalized, searchable data set that can be analyzed to produce useful information, or actionable intelligence, for dissemination to information consumers, such as incident response staff, software development teams, and IT operations teams.
Once the data has been captured and normalized, significant effort may be required to analyze it and identify anomalies that may point to a potential problem. A comprehensive data set is more likely to capture data that identifies problems, but with more data comes a larger task to normalize, filter, and organize the data into a useful form. Many organizations now collect such volumes of data as to make human analysis impractical. Software solutions are appearing to perform automated analysis, using artificial intelligence (AI) and machine learning (ML) techniques.
Analysis needs to be performed in the context of use cases. Pointing to a large data set and issuing an instruction to "discover evil" is unlikely to yield timely, relevant, and accurate results with a high degree of confidence. Use cases are developed from threat analysis to provide a working model of what to look for within a data set. For example, within the domain of authentication, individual use cases would be developed to detect indicators for irregular Windows and SSH log-ons. Each use case should be supported by filter or query strings to extract relevant data.
The dissemination phase refers to publishing information produced by analysis to consumers who need to act on the insights developed. Dissemination can take many forms, from status alerts sent to incident responders to analyst reports circulated to C-suite executives. One of the challenges of implementing this phase is to formulate intelligence in different forms for different audiences. A report written for an audience composed of other analysts will use different language and detail to one advising executive employees. Intelligence distribution can be thought of as occurring at strategic, operational, and tactical levels.
- Strategic intelligence addresses broad themes and objectives, affecting projects and business priorities over weeks and months.
- Operational intelligence addresses the day-to-day priorities of managers and specialists.
- Tactical intelligence informs the real-time decisions made by staff as they encounter alerts and status indicators.
The final phase of the cycle is feedback and review, utilizing the input of both intelligence producers and intelligence consumers. The goal of this phase is to improve the implementation of the requirements, collection, analysis, and dissemination phases as the life cycle develops. For example, feedback might address some of the following points:
- Lessons learned—What incidents occurred that threat intelligence failed to mitigate?
- Measurable success—What metrics show the success or failure of intelligence sources? One of the aims of the intelligence cycle should be to avoid collecting information for information's sake.
- Address evolving security threats—What new features of the threat landscape or the legal/regulatory landscape affect the way security and threat intelligence is collected and used?
THREAT INTELLIGENCE SOURCES
As part of the collection phase of the intelligence cycle, it is important to assess sources as they are incorporated within the data set. This is particularly important when considering threat intelligence, as this data is likely to derive from external sources. Some factors that identify the value of threat intelligence include timeliness, relevancy, accuracy, and confidence level:
- Timeliness—Threats diminish or change and evolve. Once an adversary group has been identified in an analyst's report, they are likely to try to disguise future activities and adopt different tactics. You must assess whether an intelligence source can research and disseminate updates in a timely manner.
- Relevancy—You must assess whether the intelligence produced by a source is relevant to the use cases developed for your analysis effort. For example, a threat intelligence source that focuses on Windows security is of limited use if your systems are primarily cloud applications accessed via Chrome OS workstations.
- Accuracy—In one sense, accuracy means showing that the information produced is validated and true. Accuracy can also refer to whether the intelligence is of a general or specific nature. Is it specific and accurate in the sense that you can use it to create rulesets in an automated software suite, or is it more strategic in nature? Threat intelligence is combined (or correlated) with security intelligence to produce insights that are directly relevant to your systems. For this to be successful, threat intelligence must be tagged with attributes that can be correlated to attributes in your log files and network traces. There are various schemas and frameworks for classifying threat information, which we will explore later in the course.
- Confidence levels—When a data point or analyst observation is published, the act of publishing lends the point a certain authority. It is usually appropriate to temper that authority by grading the data or analysis on some scale between reliable and unreliable. For example, the MISP Project (misp-project.org/best-practices-in-threat-intelligence.html) codifies the use of the admiralty scale for grading data and the use of estimative language for grading analyst opinion. The admiralty scale rates sources with letters from a (reliable) to g (purposefully deceptive) and information credibility from 1 (confirmed by multiple sources) to 6 (cannot be validated).
PROPRIETARY/CLOSED-SOURCE INTELLIGENCE SOURCES
Threat intelligence is very widely provided as a commercial service offering, where access to updates and research is subject to a subscription fee. Some of these commercial sources primarily repackage information coming from free public registries, while others provide proprietary or closed-source data that may not be found in the free public registries. Closed-source data is derived from the provider's own research and analysis efforts, such as data from honeynets that they operate, plus information mined from its customers' systems, suitably anonymized. Most of the commercial feed providers also market their own platform for processing and disseminating threat intelligence. There are also platform providers who do not produce their own security feeds. Some examples of commercial providers include:
- IBM X-Force Exchange (exchange.xforce.ibmcloud.com)
- FireEye (fireeye.com/solutions/cyber-threat-intelligence/threat-intelligence-subscriptions.html)
- Recorded Future (https://www.recordedfuture.com/platform/threat-intelligence)
OPEN-SOURCE INTELLIGENCE SOURCES
Open-source feeds are available to use without subscription. Open-source repositories include threat feeds similar to the commercial providers, but also reputation lists and malware signature databases. Government agencies represent one source of public threat information. The United States Computer Emergency Readiness Team (US-CERT) provides feeds of current activity and alert news, plus regular bulletins and analyst reports (us-cert.gov/ncas). US-CERT also provides a bidirectional threat feed called the Automated Indicator Sharing (AIS), available at us-cert.gov/ais. The UK's National Cyber Security Center provides similar services via the Cyber Security Information Sharing Partnership (ncsc.gov.uk). Other examples of open-source providers include the following:
· AT&T Security, previously Alien Vault Open Threat Exchange (OTX) (otx.alienvault.com)
· Malware Information Sharing Project (MISP) (misp-project.org/feeds)
· Spamhaus (spamhaus.org/organization)
· VirusTotal (virustotal.com)
While threat feeds contribute to explicit knowledge—insights that can be directly applied to a security process—you should also be aware of sources that communicate implicit knowledge. Blogs and contributions to discussion forums from experienced practitioners provide not only reporting on the latest trends in cybersecurity issues, but also invaluable insights into attitudes and instincts that contribute to success in a career as a cybersecurity professional.
** There are too many useful blog and discussion sources to include here, but the list curated by Digital Guardian (digitalguardian.com/blog/top-50-infosec-blogs-you-should-be-reading) is a good starting point. **
** While we are considering open-source or public threat intelligence feeds here, also be aware of the use of the term open-source intelligence (OSINT) to refer to a reconnaissance technique. OSINT refers to methods of obtaining information about a person or organization through public records, websites, and social media. OSINT techniques can also be a source of threat data, as researchers use them to discover more about adversary groups and malicious actors. **
INFORMATION SHARING AND ANALYSIS CENTERS (ISACS)
Since the 1990s, governments have mandated that industries where cyberattack poses risks to life or health or to national security must form public/private partnerships and industry associations to disseminate sector-specific threat intelligence. For each critical industry, Information Sharing and Analysis Centers (ISACs) have been set up. Where a generic open-source or commercial threat intelligence provider might use corporate or academic networks to gather data, ISACs produce data from their members' systems, so the data is highly industry-specific and relevant. Information shared within an ISAC is given legal protections by the PCII program operated by the Department of Homeland Security (dhs.gov/cisa/pcii-program). A list of all US-based ISACs is available at nationalisacs.org/member-isacs-3. In the UK, the Cyber Security Information Sharing Partnership (ncsc.gov.uk/section/keep-up-to-date/cisp) serves a similar purpose.
The DHS identifies sixteen critical infrastructure sectors (dhs.gov/cisa/critical-infrastructure-sectors), such as communications, energy, water, nuclear reactors and waste, emergency services, and so on. Each sector is supported by its own ISAC. One of the primary areas of focus for cybersecurity in industries that support critical infrastructure is with embedded systems and industrial control systems.
The Multi-State ISAC (cisecurity.org/ms-isac) serves non-federal governments in the US, such as state, local, tribal and territorial governments. One of the key cybersecurity concerns for governments is interference in the electoral process and the security of electronic voting mechanisms. In fact, there is an ISAC dedicated to election infrastructure security issues (cisecurity.org/ei-isac).
Healthcare providers are targeted by criminals seeking blackmail and ransom opportunities by compromising patient data records or by interfering with medical devices. The Health ISAC is at h-isac.org.
The financial sector is an obvious target for fraud and extortion. Attackers can target both individual account holders and financial institutions themselves. Serious financial shocks, such as major trading platform or ATM outages, can also pose a national security risk. The Financial Services ISAC is at fsisac.com.
As with most commercial industries, the aviation industry is targeted for fraud, but there are also substantial risks from terrorists or hostile nation-state actors seeking to disrupt services or cause casualties. Air traffic control and the safe operation of aircraft depends on many interconnected systems, some of which use aging infrastructure or technology that is susceptible to interference and spoofing, such as radar and GPS. The Aviation ISAC is at a-isac.com.
What do these steps look like from the pen tester's/hacker's perspective?
What do these steps look like from the security analyst's/defender's perspective?
Locard's Exchange Principle
The perpetrator of a crime will:
- Bring Something to the crime scene
- Leave something at the crime scene
Map controls to IoC (Indicators of Compromise)
Focus on esstional resources
- Servers (application level)
- End users (logins, privilege escalation)
- End points
- Obtain information about
- Indicators of Attack (IoA)
- Indicators of Compromise (IoC)
- Apply logic and specific conditions
- Identify courses of action
Information Sharing Ecosystem
- Information Sharing and Analysis Center (ISAC) - bigshot
- Two-way: Gather and disseminate information
- Organized by critical infrastructure sector
- Information Sharing and Analysis Organization (ISAO) - less organized
- Not organized by sector
- Less exclusive
- Fewer legal restrictions
- Organized by sector
- Must conform to minimum standards
- But less restrictive than an ISAC
STIX - Structured Threat Information eXpression
- Standardized language for describing threat information
- Developed by MITRE
- OASIS Cyber Threat Intelligence (CTI) Technical Committee
- Used by various Intelligence Sharing (ICS) entities
TAXII - Trusted Automated eXchange of Intelligence Information
- Defines how cyber threat info can be shared between
- Message exchanges
- Three models
- Source / subscriber
- Hub and spoke
- Source / subscriber
- Yeti - an open source TAXII implementation
Sharing categorized threat intel via TAXII
- Push and pull information
- By category
- By threat
- Organizations can then ingest that intelligence
- Sharing with groups
- Organizations with a TAXII client can push information to TAXII servers
- Groups trust the source
- Private groups can exist
- TAXII Clients
Joining a threat Feed
- Download the STAXX Client
- Configure your data sources
- Set up your download schedule
- Find a service
- Set up a client (e.g., TAXII)
- Available for download (e.g., YETI)
- Purchase or otherwise obtain a PKI certificate from a commercial provider
- Provide additional information (e.g., IP address, sign agreements).
- Connect and get info!
- Customize and share!
THREAT INTELLIGENCE SHARING
As well as identifying timely, relevant, and accurate sources of threat intelligence, you need to consider use cases for making that data actionable as it is disseminated to different intelligence consumers. Threat intelligence can be used to improve capabilities across different security functions.
Risk Management and Security Engineering
Risk management identifies, evaluates, and prioritizes threats and vulnerabilities to reduce their negative impact. Security engineering focuses on the design and architecture of hardware, software, and network platforms to reduce their attack surface. Strategic threat intelligence is important for establishing an up-to-date model of threat sources and actors, and their motivations, capabilities, and tactics. This model can be used as part of a risk management framework and security engineering to select and deploy new technical and administrative security controls, or to improve the configuration of existing controls. Threat intelligence should be shared with network and application operational security teams so that they can apply best practices to the controls that they have responsibility for. For example, threat intelligence can provide information about new vectors for attacking application code. It is important for this information to be shared with software development teams so that they can adopt suitable secure coding practices in response.
Where risk management and security engineering make best use of strategic insights, incident response is better served by operational and tactical insights. For example, the analysis benefit of tactical threat intelligence is to allow you to pivot from a data point, such as a suspect DNS domain in a web access log entry, to information about that domain on a reputation list, and whether it is associated with specific malware tools or adversary groups.
At a strategic level, threat intelligence can identify previously unrecognized sources of vulnerabilities, such as embedded systems, Internet of Things (IoT) home automation devices, deep fakes (https://www.securityweek.com/deepfakes-are-growing-threat-cybersecurity-and-society-europol), or AI-facilitated fuzzing to discover zero-day vulnerabilities (threatpost.com/using-fuzzing-to-mine-for-zero-days/139683). At an operational level, threat intelligence can identify priorities for remediation, such as a campaign targeting a vulnerability in web server software. Threat intelligence can also provide ongoing monitoring and analysis of vulnerabilities such as Meltdown and Spectre (securityintelligence.com/spectre-meltdown-and-more-what-you-need-to-know-about-hardware-vulnerabilities), which could pose lasting risks well past the impact of their initial announcement.
Detection and Monitoring
Acquiring accurate and relevant information about attacks suffered by organizations working in similar industries will improve automated detection and monitoring systems, though there will be some increased risk of false positive alerts and notifications. Adding more rules and definitions based on observed incidents to automated tools will create more chances for malicious indicators to be matched (true positives). Unfortunately, it also creates more chances for non-malicious data points to be matched as suspected indicators (false positives).
As well as improving operational capabilities, threat intelligence promotes new strategic approaches to information assurance, such as proactive threat modeling and threat hunting techniques, which will be the subject of the next lesson.
Front of Flashcard 1 of 3
Your chief information security officer (CISO) wants to develop a new collection and analysis platform that will enable the security team to extract actionable data from its assets. The CISO would like your input as far as which data sources to draw from as part of the new collection platform, worrying that collecting from too many sources, or not enough, could both impede the company's ability to analyze information. Is this a valid concern, and how can it be addressed within an intelligence life cycle model?
Back of Flashcard 1 of 3
Yes, it is a valid concern. The requirements (or planning and direction) phase of the intelligence cycle can be used to evaluate data sources and develop goals and objectives for producing actionable intelligence to support use cases demanded by intelligence consumers. You can also mention that the feedback phase of the cycle provides the opportunity to review sources and determine whether they are delivering valuable intelligence.
Front of Flashcard 2 of 3
What are the characteristics to use to evaluate threat data and intelligence sources?
Back of Flashcard 2 of 3
Firstly, you can distinguish sources as either proprietary/closed-source, public/open-source, or community-based, such as an ISAC. Within those categories, data feeds can be assessed for timeliness, relevancy, and accuracy. It is also important for analyst opinions and threat data points to be tagged with a confidence level.
Front of Flashcard 3 of 3
What are the phases of the intelligence cycle?
Back of Flashcard 3 of 3
Requirements (often called Planning and Direction), collection (and processing), analysis, dissemination, and feedback.