The Problem in Limiting a Penetration Test

When approached by a business recently we were given a list of servers that the company did not want us to audit during the penetration testing phase of the audit. While this is not uncommon the shear number of servers on the list raised some questions. Most of the servers at the organization were billing systems and systems that stored customer records. Normally this is exactly the type of servers we want to check to make sure they are secure. The business justification in this case was that the servers were too sensitive to the corporate operations and that if any of the servers were inadvertently taken offline during the penetration testing that the company would lose money to the tune of up to $1 million per hour.

After completing the rest of the audit we had a sit down meeting with the management team and they would not budge on the rules of the engagement. In addition to this limitation they also stated that we were not to test any of their phone system during the audit. We agreed to play by their rules but we reminded them that hackers don't have to play by a set of rules and that by not testing these systems they would not know if there were problems that may be identified by the enemy at a later time. The company decided to assume the risk.

17 days after the penetration testing was completed the company detected strange activity on one of their systems. We were asked to do live forensics on the system and we immediately detected applications running in memory that were not supposed to be there. Luckily through troubleshooting we were able to confirm that the files in memory were not malicious but the increased memory utilization of the update processes (for a very popular program) were the cause of large amounts of memory being available on the system to the critical applications that were running on the system. 4 days after this discovery they company requested that the additional systems be audited.

I want to remind companies that hackers don't play by rules. They don't care if a critical system crashes or if your customer database is taken offline. In fact that's the very goal of some hackers out there. The only thing they want to do is disrupt your business and some hackers will take that as a win. When doing penetration testing it's important for the team to test systems as safely as possible but the risk of having a system go offline during testing is a real concern for business. It would be best to have this occur in a controlled maintenance window as opposed to a busy time of the day where those systems are bringing in revenue.

If a company limits the scope of testing they also limit the usefulness of the audit report. It's not possible to have the best of both worlds in this case and a controlled crash is a much better outcome than a loss of income due to a hacker discovering an unpatched system during a critical time. Get the most of your audit and make sure you have all areas tested completely and documented accurately. Then if something happens in the future at least you will know the risk associated with a particular system or platform.

If you are dead set on setting rules set time based rules for the audit. Review the test plan and understand what the auditors will be doing to test your security and at a minimum don't change your day to day operations just because you are going through an audit. You want to make sure your team is adequately prepared to deal with the results whatever they may be.