Saturday, 22 December 2012

Test-Driven Security

For many years, security coding has been a problem in many companies. So far, the following solutions have been implemented with more or less success:

  • teach people a list of dangerous functions and how to avoid them or use them securely. This method mostly comes from the old "don't use strcpy" mentality.
  • teach people what type of vulnerabilities exist and how to prevent them. We can call that the "OWASP top10" method. 
  • Only hire the best developers. We can call that the "successful startup" model. However there is only a limited number of best developers and everyone does mistakes.

With more and more frameworks and all the protection in place (automatic data mapping, XSS protection, automatic CSRF protection). The traditional issues are less and less relevant (if you already have a SDLC and care about security, this obviously does not apply to people getting a PHP website done by a trainee in 2 weeks). Most of the issues are now based on business logic and corner cases.

Test-Driven Security is another really simple method: instead of asking developers to write secure code,  this method just explain how to test if their code is secure and tell them to test the security of their code like they will do for any other functions. 

The problem with tests is that most people only do "positive testing", something works ("WINNING") or does not. Test-Driven Security is based on "negative testing" and make sure that things cannot happen: testing that things do not work.

For example, if you want to test authentication, you should add the following test cases:
 - Can I log in without password: it sounds silly but how many of you are 100% sure that your application is not vulnerable to this (especially if you have a LDAP backend, try to remove the password parameter from your login request) ?
 - Can I log in with the wrong password?
 - Can I log in with only the first 8 letters of my password ?
 - ...

You can do the same thing for authorisation: 
 - Can I access someone else personal information
 - Can I edit someone else personal information 
 - Can I reset someone else password
 - ...

Another point is testing for "Breaking the syntax", that I already covered in a previous post.

The good thing about adding all these checks inside your tests is that these checks will be performed as often as you run your tests... and you will be 100% sure that these security questions are covered, and that no regression happens between two releases of your application.

I did a talk on this few months ago at OWASP Melbourne, and most of the questions were around the return of investment of this method. To be honest, that's a really good question and I have no metrics on this. But I think that it's not too much work to "teach" people how to do that and get 100 test cases written to cover the basics, especially if you compare that to the price of a pentest...


Anonymous said...

Correct solution:

Use a framework which hides the details from you. It protects against mistakes and bad developers. It also makes things uniform.

Louis Nyffenegger said...

Yes, but it won't solve possible authorisation or authentication issues :/ said...

Liked it