by Pen Testing Standards and Frameworks
Several sets of standards and frameworks have been developed to provide a common base of understanding and expectation for pen tests. Some examples have been listed below.
CHECK framework
- Developed by the Communications-Electronic Security Group (now the National Cyber Security Centre), which is part of the UK Government Communications Headquarters. This scheme ensures that government agencies and public entities can contract with certified companies to identify vulnerabilities in their confidentiality, integrity, and availability (CIA) by testing their networks and other systems.
The Open Web Application Security Project (OWASP) Testing Framework
- Developed by a multinational organization that collects and shares security practices with software developers, this framework provides pen testing and other testing techniques for each part of the software development life cycle. For more information, refer to www.owasp.org.
Open Source Security Testing Methodology Manual (OSSTMM)
- Developed by the Institute for Security and Open Methodologies (ISECOM), this document is a peer-reviewed guide to security testing and analysis that enables you to tighten up operational security. For more information, refer to www.isecom.org/research.
Penetration Testing Execution Standard (PTES)
- Developed by security service practitioners to provide business professionals and security service providers a basic lexicon and guidelines for performing pen tests. The PTES is the general standard, while detailed information is provided in the PTES Technical Guide. For more information, refer to www.pentest-standard.org.
NIST SP 800-115
- Developed by the US National Institute of Standards and Technology (NIST), the Technical Guide to Information Security Testing and Assessment provides practical recommendations for designing, implementing, and maintaining pen test processes and procedures.
Processes Commonly Used for Pen Testing
- Planning: Most recognized pen test processes include a planning phase. Depending on the authority that promulgates the process, this phase might also include identifying the scope of the engagement, documenting logistical details, and other preliminary activities that need to occur before the commencement of the pen test.
- Reconnaissance: In the reconnaissance phase, the tester gathers information about the target organization and systems prior to the start of the pen test. This can include both passive information gathering, such as collecting publicly available information about the organization, and deliberate acts, such as scanning ports to detect possible vulnerabilities.
- Scanning: The scanning phase is generally a bit more in depth than the reconnaissance phase. This is where vulnerability assessment begins. Static and dynamic scanning tools evaluate how a target responds to intrusions.
- Gaining access: This phase is when the actual exploit begins, by applying the information gained by reconnaissance and scanning to begin to attack target systems.
- Maintaining access: In this phase, the pen testers install mechanisms allowing them to continue to access the system. This phase is also where pen testers reach deeper into the network by accessing other network systems.
- Covering tracks: This phase concentrates on obliterating evidence that proves an exploit occurred. It generally consists of two facets: avoiding real-time incident response efforts and avoiding post-exploit forensic liability.
- Analysis: In this phase, the pen tester gathers all the information collected, identifies root causes for any vulnerabilities detected, and develops recommendations for mitigation.
- Reporting: The reporting phase is where the information from testing and analysis are officially communicated to the stakeholders. Although reporting requirements can vary due to customer needs or statutory regulations, most pen test reports list:
- Vulnerabilities detected.
- Vulnerabilities exploited.
- Sensitive data accessed.
- How long the pen tester had access.
- Suggestions and techniques to counteract vulnerabilities.
Communication and the Pen Testing Process
As with any type of review, whether internal or for hire, communication between the testing team and the stakeholders is of paramount importance. All facets of communication need to be evaluated and decided upon prior to the pen testing engagement, such as:
- The communication path, or chain of command. In a pen testing situation, it’s equally as important to ensure that the right people are informed as to what information should be shared. For instance, the organization might not want all staff to know when a pen test is occurring, particularly if they want to check on the effectiveness of using social engineering tactics to penetrate a network. The client IT manager and CIO/CISO should be aware of the engagement. Additionally, some key department managers should also be aware in case unforeseen incidents might affect their departments.
- Communication with client counterparts. The designated lead of the pen testing team should have close communication with their client counterpart (typically the IT manager). To reduce possible confusion, all communication between the pen testing team and the client should go through this point of contact. The two lead roles must both be hands-on. This allows for immediate response in case of incidents, unexpected discoveries, additional client requests, or anything else that might lead to extended time or scope creep.
- Communication within the pen testing team. The pen testing team should have internal communication protocols as well. For example, sub-teams working on specific tasks should apprise the lead of their progress. They should inform the lead immediately of unexpected findings, such as evidence of prior security breaches or if they discover current hacking activity. The lead will then contact their client counterpart to discuss what should be done.
- What information to communicate, and when. What should trigger official communications? Describe any standard process stages, such as the planning and reporting stages, that require meetings to be held. Also describe the actual deliverables, such as status and interim reports, as well as the final report to be provided. What about “show stoppers” or other critical findings? The pen test team must be able to prioritize findings as they occur and identify findings that are urgent enough to trigger special communications. When a pen tester encounters evidence of a compromised system, should the Incident Response Team be notified to ensure that the organization is aware of the attack? If the evidence appears to be “fresh,” the pen test might need to be suspended until the security breach is handled. If it is historical, the pen test team should log the discovery and continue with the task at hand.
- Regular progress briefings within the team. If different members of the pen test team are conducting simultaneous attacks, there should be internal coordination to ensure team members are not accidentally interfering with each other. The lead might opt to have daily “scrum” type meetings, in which each member describes what they did yesterday, what they will do today, and identify anything blocking their efforts. The lead or project manager can then allocate resources or request conflicting activities to be temporarily suspended.
- Regular progress briefings with the client. If the pen test will take more than a few days, the client might want regular progress updates. This can be done weekly or as deemed necessary. Keep in mind that “the client” is probably not just one person but could be several managers who need to remain in the communications loop. The client may request that these managers each directly receive a copy of status updates, or they may request that reports are given to only one representative, who will internally distribute copies. Typically, the final report is given to a single party as part of a formal handoff. In some cases, certain findings may be too sensitive to share with all on the approved recipients list. However, this is more likely to be the exception rather than the rule. Having a clear communication path will ensure that all relevant parties receive reports in a timely manner. Emergencies would be handled separately, though ongoing issues such as client interference, delays, or other problems should be raised at status meetings.
- Clear identification of the reasoning behind communication activities. Consider how a situation might need to be addressed if the pen test attempt is detected. It is possible that several testers might focus their efforts on a key system at the same time, thus making the breach debilitating or quite obvious. In such a case, the testing team might need to work together to scale back on their efforts to de-escalate the effects of the test. Providing situational awareness to key client personnel can also help deconflict the breach, enabling the pen test to continue so that additional issues can be found, exploited, and analyzed.
- Possible adjustments to the engagement. The nature of a pen test is that it is a fluid process. Information that is discovered during the reconnaissance phase drives the decisions on what exploits to try and, ultimately, what solutions to propose. Awareness of the need for contingency planning for the pen test engagement itself enables you to incorporate it into your plans and to re-prioritize the goals of one activity or large sections of the pen test.
- Disclosure of findings. It is incumbent upon a company to fully disclose vulnerabilities and breaches to their customers, suppliers, regulators, or members of the public who may be harmed by the breach. If you, the pen tester, were paid to help discover those vulnerabilities and breaches, any findings should be strictly confidential for both legal and ethical reasons. An exception to this could be if you uncovered criminal conduct, in which case you might be obligated to notify law enforcement. If a question arises regarding disclosure of findings, even if disclosure would be for the general public good, it is not the pen tester’s job to make that decision. You should consult with your team’s legal counsel in such cases.
Contract Types
Rules of Engagement
In pen testing, the rules of engagement is a document or section of a document that outlines how the pen testing is to be conducted. They describe the expectations of the client and the rights and limitations of the test team.
Some facets of the rules of engagement are described in this table.
| Component | Description |
| Timeline | The timeline of a pen test engagement is a clear enumeration of the tasks that are to be performed as part of the engagement, and the individuals or teams responsible for performing those tasks. As the engagement progresses, stakeholders can use the timeline as a progress indicator, and adjust it as needed during the engagement to account for any unexpected events. The timeline is often shared with stakeholders in a Gantt chart format. |
| Location of test team | The location of the test team in relation to the client organization needs to be agreed upon. Depending on factors such as how many locations an organization occupies, whether or not remote installations are in different nations, and what sort of remote technology is available to access multiple locations, the parties should agree and record the amount of travel required, if any, to conduct the pen test. |
| Temporal restrictions for testing | When the actual test begins, are there constraints on the days and times that the testing can be performed? |
| Transparency of testing | At the client organization, who will know about the pen testing?For the test team, what information will be provided prior to the start of the engagement? |
| Test boundaries | What’s being tested, and what is not?Define the acceptable actions, such as social engineering and physical security tasksIf invasive attacks, such as DoS attacks, are part of the testing, are there any restrictions on their use? |
Types of Assessment
There are several general categories of assessment that an organization can use to help define the scope of a pen test engagement.
- Goal-based or objective-based assessments provide focus points for the pen test. They begin by the client listing the items or information that needs to be protected. Then, the pen test team will develop plans to obtain the goal or objective through any attack techniques available. This approach closely mimics the attacks that might be launched by a malicious party, so they can provide significant ROI for the client organization.
- Compliance-based assessments are government- or industry-required tests based on an established compliance framework. Common US compliance frameworks include PCI DSS, DISA STIG, FEDRAMP, and FISMA.
- Red team assessments test an organization’s detection and response capabilities by emulating a malicious actor who targets attacks and avoids detection. The red team accesses sensitive information any way it can without being caught in the act. Red teams usually have longer time frames in which to work. Because stealth and avoidance is of great importance to the red team, they function more like an advanced persistent threat (APT), keeping a low profile while infiltrating the network. By contrast, pen test teams are usually time constrained and often cannot afford to be as patient. As such, they may be “noisy,” while red teams are “quieter.”
Color Teams
The idea of color teams evolved from military readiness exercises. In general, red teams attack, while blue teams defend. In some instances, purple teams help coordinate interactions between red and blue teams, and in other cases, white teams establish rules and monitor the testing.
Types of Targets
| Target Type | Description | Attack Considerations |
| Internal | Assets can be accessed from within the organization. Internal attacks might be caused by malicious insiders or by external hackers who have gained credentials through a phishing attack. | An excellent candidate for all attack types IF direct access to the internal network can be established. |
| On-site | Asset is physically located where an attack is being carried out. | Accessibility depends on controls at the site. A physical attack might go undetected at a large facility with many people. Centralized resource locations will probably have more points of entry and attack vectors to choose from. |
| Off-site | Asset provides a service for an organization but is not necessarily located at the same place. | An organization’s remote offices and satellite locations are less likely to have as many security controls as headquarters. As such, an attacker would lose the “cover” of anonymity. However, lax security might still make it possible to carry out physical, Wi-Fi, or possibly remote access/VPN attacks. You would have to assess if the remote location is worth the effort. If the remote location does not itself house interesting assets, it might provide a back door (such as an unguarded WAN or VPN link) to the main facility. |
| External | Asset is visible on the Internet, such as a website, web application, email, or DNS server. | Not a good candidate for attacks (such as sniffing or ARP poisoning) that require direct access to the network segment. |
| First-party hosted | Hosted by the client organization. | Might be easier to attack than third-party hosted services, as most companies do not have the same resources, expertise, or security focus as a provider. |
| Third-party hosted | Hosted by a vendor or partner of the client organization. | Not impossible targets, but established providers are more likely to have good controls in place. Smaller, newer hosting companies may have fewer resources and less security expertise. These might be easier to attack than larger, more mature providers. All third parties can be vulnerable to zero-day attacks. |
| Physical | Can include the client organization’s premises or any physical device belonging to the client organization. | Physical attacks are an excellent way to plant sniffers, remote-controlled devices, keyloggers, and other attack tools in the private network. |
| Users | They generally have access to resources that might be restricted to outside parties. | Users are usually the easiest attack vector because they are so susceptible to social engineering. |
| SSIDs | Can be targets when an attacker is attempting to access a wireless network. | Evil twins and other Wi-Fi attacks require close physical proximity to the premises. |
| Applications | Can be targets, as they are often linked to sensitive data such as credit card numbers. | You’ll have to determine which applications are in use. If it runs in user context, you’ll want to escalate privilege once it is compromised. |
Specialized Systems
General Considerations
| Consideration | Description |
| Organizational policies | Organizational policies are formalized statements defining how the organization intends to meet its long-term goals. They can cover numerous topics, including security, privacy, compliance, and acceptable use of resources. Pen test engagements should be designed so as to be in concert with existing organizational policies. |
| Security exceptions | Some organizations provide ways to apply for policy exceptions, where certain organizational policies are not enforced for identified technologies or resources. Existing security exceptions should be identified as being either within or outside of the engagement’s scope. |
| NAC | Network Access Control encompasses the collected protocols, policies, and hardware that govern if and how devices can connect to a network. If a device can pass a health check, it can connect to the network. Devices are agent-based or agentless. |
| Whitelisting and blacklisting (IPS/WAF whitelisting) | Whitelisting blocks all users or IP addresses except those included on the whitelist, while blacklisting allows all users or IP addresses except those included on the blacklist. These practices are commonly used with intrusion protection systems (IPSs) and web application firewalls (WAFs). It is generally recognized that implementing whitelists is more restrictive and thus more secure than implementing blacklists. |
| Certificate and public key pinning | Certificate and public key pinning is the process of associating a host with its expected X.509 certificate or public key. Pinning bypasses the certificate authority (CA) hierarchy and chain of trust to lessen the impact of man-in-the-middle attacks. Used in securing wireless channels, the act of pinning a certificate or public key helps guard against vulnerabilities in well-known protocols such as VPN, SSL, and TLS. |
Guidelines for Scoping and Negotiating Pen Test Engagements
Here are some guidelines for scoping and negotiating pen test engagements:
- Determine the types of assessments you want to conduct:
- Goal-based or objective-based
- Compliance-based
- Red team
- Clearly define the end goals of the engagement.
- Determine what testing strategy you need to use:
- Black box
- Gray box
- White box
- Determine what types of threat actors you want to emulate, and what their capabilities and intent might encompass.
- Consider recommending that the client organization engage in some threat modeling so that their objectives and expectations can be clearly defined.
- Identify all targets, whether conventional or specialized systems, and the risk tolerance associated with each.
- Be sure to account for existing controls and scenarios.
- Existing organizational policies and security exceptions
- Existing whitelists and/or blacklists
- The use of certificate and public key pinning
- The use of NAC devices and controls
- The need for premerger or supply chain security testing
- Create, maintain, and adhere to a comprehensive schedule.
- Find ways to avoid scope creep; consider including disclaimer language to protect the test team from any adverse events resulting from allowing or denying scope creep.
- Consider using a scoping checklist to gather information that will help shape the boundaries of the engagement.
- Identify each deliverable, including all documents and meetings.
Activity Assignment and Sequencing
As with other types of projects, a penetration test should be carefully managed. This includes sequencing tasks and assigning resources to meet the schedule and objectives as outlined in the statement of work. Penetration testing, however, can be more fluid and dynamic than other types of projects. The direction of the investigation will evolve depending on findings. Teams need to be flexible and respond to changing conditions. Follow these best practices when assigning and sequencing activities in a penetration test:
Start with initial task sequencing based on these common pen test stages:
- Passive reconnaissance
- Active reconnaissance
- Vulnerability assessment
- Penetration
- Exploitation
- Post exploitation
- Fit in non-technical tests such as social engineering and physical attacks at the earliest opportune moments.
- Whenever possible, “front load” the test with as many early assignments as possible to leave extra time at the end for the unforeseen.
- Give extra time to activities that are opportunity dependent (such as social engineering and physical attacks) or evasion-oriented (such as slow vulnerability scans).
- Be prepared for findings to spawn new investigations.
- Ensure that all investigations are driven by the requirements.
- If you are training new pen testers, pair less experienced team members with more experienced testers unless that pairing might endanger the mission of a particular activity.
- If a team member uncovers a serious problem that is outside the scope of the pen test, present the findings to the client and ask the client what they would like to do. Do not expand the scope of the investigation unless permitted by the SOW.
- When you have your initial assignments and sequencing ready, call a tactical meeting to outline the plan to the team. In some cases, an experienced team might self-organize, and collaboratively determine sequencing and task allocation.
Risk Responses
Escalation Path for Communications
Good communication is essential for the success of the penetration test. Not only must the pen test team be able to communicate amongst themselves and with their lead, but the team lead must also be able to communicate with the designated client contact. Having an escalation path for communications protects individual pen testers from having to make risky or potentially damaging decisions on their own. You also want to make sure that communications follow a chain of command, and that team members report and escalate issues only to authorized individuals. Use these best practices when establishing an escalation path for communications:
- Establish a clear chain of command in the pen test team. Make sure that communications follow that path.
- Make sure that the pen test team project supervisor has a counterpart on the client side that they can immediately bring issues to.
- Ensure that there is always a supervisor on duty, including a fail-safe operator, for team members to contact.
Agree upon thresholds and protocols for contacting the other side during a problem, including:
- When/how the client will notify the pen test team that a test is unacceptably interfering with operations/system performance.
- When/how the pen test team will involve the client IT department if an accident occurs or a system becomes destabilized or unresponsive.
Train all team members to:
- Check in regularly with their lead, especially when starting and finishing a task.
- Check in with their lead if they encounter anything unusual or outside the scope of their task.
- Not make any decisions outside the scope of their task without authorization from their lead.
- Alert their lead immediately if a problem arises.
Guidelines for Preparing for a Pen Test Engagement
Here are some guidelines you can follow when preparing for a pen test engagement.
- Ensure that your team is well trained in the tasks they will undertake.
- Make sure there is a clear chain of command with a clear communications path.
- Train the team to consult their supervisor when confronted with an unexpected situation or decision.
- Pair less experienced testers with more experienced ones unless it puts the activity at risk.
- Ensure that the client’s IT department (at least the managers) is aware of the test, and that they have good backups and contingency plans to restore affected systems.
- Train your team to stay within the scope of the engagement unless authorized to expand their investigation.
- Train your team to log evidence of previous or existing malicious activity, to continue with what they are doing, and to escalate findings for further instruction.
- Ensure that the team fully documents their steps, collects as much data as possible, and uploads this information to a central repository for analysis.