Many providers now offer time boxed penetration testing, often without explicitly stating it, which can leave clients unaware of its limitations.

This article aims to define what time boxed penetration testing is and shed some light on how Project Black approaches time boxed engagements.

What is Time Boxed Testing?

In traditional web application penetration tests, we aim for exhaustive testing - our goal is to achieve comprehensive coverage of all web application components.

This approach can be time consuming making it less feasible when limited budgets are involved.

In contrast, time boxed penetration testing is structured around a fixed duration. Instead of focusing on achieving full coverage, the goal is to make the best possible use of the time allocated to identify key vulnerabilities in high-risk areas of the application.

How is Testing Prioritised in Time Boxed Engagements?

Testing in a time boxed engagement is prioritised in three main ways.

High Risk Areas

High risk areas of a web application are functionalities that, if found vulnerable, could have a significant impact on security. These areas therefore make sense to prioritise first in time boxed engagements.

This includes authentication and authorisation mechanisms to prevent unauthorised access, privileged functions like file uploads, and complex workflows where correct business logic is essential.

Client Specific Priorities

You know your application better than we do which means you might be able to help us with prioritising testing efforts.

Practically, this could mean concentrating our efforts on parts of the application with past incidents, lower code quality, or newly added features.

Sampling to Achieve Breadth

Development patterns, such as consistent coding practices or reusable internal libraries, can allow us to take shortcuts by testing representative functions in portions of applications.

For example in a microservices based application, instead of assessing every API endpoint exposed by a microservice, we may instead test a sample of endpoints. This ensures we achieve breadth of coverage across all core application microservices.

The Outcome

The outcome of applying these three principles—focusing on high-risk areas, addressing client-specific priorities, and using sampling for breadth - results in a targeted testing approach, as illustrated in the simplified diagram below.

By applying this strategy, we can concentrate efforts where they matter most to identify high risk vulnerabilities.

This is in contrast to a traditional approach to testing which aims for complete testing coverage.

Limitations

Are there limitations of this approach? Definitely.

• Incomplete Coverage: The effectiveness of sampling relies on the assumption that consistent development patterns are used throughout the application. However, if these practices are not in place, sampling may fail to find unique vulnerabilities across varied implementations of repeated functionality.
• Misidentification of Priorities: Everyone has unknown unknowns. When client priorities guide testing, there is the potential for assumed "safe" areas to be overlooked. "Stable" components might contain vulnerabilities that go undetected.