Wednesday, 5 December 2012

How not to get a good pentest?

As an introduction, you should probably read "Maker's schedule, Manager's schedule" (don't be afraid by the startup ad it's a really good read) for more inside on how pentesters (makers) (should) work.

I put together a list of things that will ensure you don't get a good pentest:
  • Make sure a WAF and/or IDS are used and don't put the testers in a white-list.
  • Organise a lot of meetings, the rule is "the more meetings... the better".
  • Make sure you don't get a technical person involved during the scoping of the test/application.
  • Make sure you put multiple layers of people between the testers and the developers/sysadmins, if you don't they may be able to have all information in time and move forward quickly... It works for the remediation part as well.
  • Ask for information (full write-up) on discovered issues as soon as they appear and make sure you fix them as soon as possible. This way testers won't find more in-depth issues in your system and the down time due to fixes will ensure they can't work.
  • Don't provide access to source code if needed. EVER. 
  • Make sure some performance tests are performed during the security testing... nothing like a really slow environment when you're doing security testing.
  • Make sure there is no sample of data provided and that the databases are as empty as possible
  • Ask for daily updates, or even better twice a day... (see maker's schedule above to understand this)
  • Make sure you don't provide administrator/root access during host review. 
  • Don't provide internet access to testers... They may need to search for something or download a tool.
  • Make sure the first day is spent waiting for credentials or waiting for the environment to be setup.
  • Make sure test and production environments are as different as possible.
  • Make sure the environment is constantly changing, new features added, the application can't look the same on the first day and the last day.
Any other ideas?


Matthew L. said...

I'd also recommend using a wonky web-based VPN solution (split-tunnel, of course) to get to the client's target machines as they don't want "outsiders" in their environment and because paying $3000 for travel will apparently destroy them.

Anonymous said...

Make sure your perimeter is as limited as possible (one VirtualHost is usually more than enough). Pentesters may use a gigantic flaw in another VHost to compromise your server.

Anonymous said...

I highly disagree with your first point. The poing of a pentest is to accurately simulate an attack. Are you going to disable your IPS/IDS for a legit attacker? No. You're not. So why would you for a pentester?

Louis Nyffenegger said...

It depends, if you want something more like a red team pentest, keep the IDS/IPS. If you want people to find all the vulnerabilities in your site, you probably want to whitelist them.
A pentest is a time limited exercise, if you waste one day because you're blacklisted, your clients are wasting money in my opinion.
If you do your pentest based on a year-long engagement (testing all the time), keep the IDS/IPS as well.

Tom said...

All testing must be done on-site for no good reason.

"I really want to learn what it is that you do."

Louis Nyffenegger said...

We are just going to get X to sit with you during the testing so he can learn from you...

Nadeem said...

Make sure the web server running the application is given the worst possible resources and make it super slow so that the pen-tester feels like shooting himself in the head after the first hour of testing.

Anonymous said...

Great! That all is soooo true. :-)

Others have the same problems.