Preparing for your assessment

Sufficient preparation is key to getting value from your security engagement. It was astounding as a consultant how many clients spent thousands of dollars after the kickoff having us wait for access or documentation. Preparation has multiple components that contribute to getting your vendor up and running.

Logistics

The first set of requirements are organizational:

1. Internal alignment: Security services engagements can be fraught, especially if they are allowed to turn adversarial. Before the vendor starts, make sure you garner buy-in from stakeholders across your organization. A crucial element of this communication should be to warn the blue team, unless it is a covert assessment. So long as testing the organization’s response to an attack is not a goal, avoid putting unnecessary strain on your coworkers.

2. Define and document a single communication channel for the consultants to interface with your organization. This will ensure you can easily track progress, respond to questions, and manage dispatch into the rest of your company. You should also setup an escalation policy and channel. For example, you might want critical issues reported immediately and fed to your vulnerability management process.

3. Known risks: Consider documenting and sharing your internal risk assessments, threat models, and previous assessment results with your vendor. This level of transparency is uncommon, but can substantially improve their ability to hit the ground running and speak directly to your most business-critical risks.

Technical preparation

Technical preparation is also necessary before your vendors can start any assessment. For a deep dive into why, I recommend Jerome Smith of NCC Group’s The Why Behind Web Application Penetration Test Prerequisites.

One often overlooked step is to resolve any low hanging fruit and outstanding issues before commissioning a new engagement. Especially for anything beyond a vulnerability scan, you should preemptively conduct a scan and resolve obvious issues. This may seem redundant with the services you’re commissioning, but it is a way to optimize the engagement value. Otherwise, your vendor will spend time reporting these findings at a much more expensive cost-per-bug. Do you want to pay someone to tell you things you already know?

Generally, network assessments can make it obligatory to test in production. Where possible though you should instead set up a test environment. This environment should duplicate production, including infrastructure and integrations. This is most important for security sensitive features, such as single-sign on, or those called out in threat models and risk assessments. Additionally, make sure you properly configure and enable relevant features. Provide access to all possible roles (including internal ones) and pre-populate test data where possible.

That last item is fairly uncommon, but do you want to be spending consultant day rates on “true positive pentest noise”?

On a related note, to the extent possible you should institute a change freeze on the target during the test period. This will avoid any disruption to testing and can clarify (via versioning) root cause analysis.

Finally, be sure to disable undifferentiated or out-of-scope defense in depth controls.

Avoiding or bypassing these controls can consume significant time. That provides no value when they are not the target. Some common examples include web application firewalls, IP allowlisting, and risk-based authentication. However, be sure to document the disabled controls so they can be considered when assessing the risk of findings.

Onboarding consultants

There are several important components of onboarding that will make your vendor’s personnel most effective. This should start with any hardware and software needed for their task. It can be efficient to start from your standard employee onboarding and profile. For vendors, it is also more likely you will need to facilitate remote access. The larger your organization, the earlier you may need to start this process. In enterprises, it can require involvement from legal, human resources, and IT. Beyond general setup, you should also ensure you get test accounts and access provisioned. Any additional demos, developer and customer documentation, and code you can provide can expedite and increase the depth of coverage of the engagement.

Tips from the survey

Here’s what other professionals recommend: