Thursday, 31 May 2012

Reporting automation... more details...

When I first talked about reporting automation, I forgot to mention a major point. And since I have been "talking on twitter" about it with Dradis authors (If you have report automation issues you should totally check Dradis, it's pretty cool).

I didn't talk about: Information storage, i.e.: the way your issues and report is going to be stored. And that's in my opinion one of the most important point.

When you're a pentester, you're working:
  • off site in the office
  • on site for a week at a client's office, 
  • then you do a wireless test and spend a day without being connected to the network
  • then you setup a crazy network to MITM an iphone application and don't really want to break it
And in all these situations, you will need to be able to read and write issues...

When you're a pentester, your report:
  • need to be reviewed by a tech person
  • need to be reviewed by a business person 
  • and mostly you want your co-worker to write a good part of it ;)
All these situations create a really big problem of data synchronisation, especially when you're in a rush...

I often see people get a tech review of their issues write-up while they write the exec summary, then they copy/paste the changes from the review.


If your reporting solution is based on a single Word document or a centralised database... You just lost the productivity game...

Dradis solves this problem by allowing you to import sqlite databases to your current project... That's a good way to do it but there is so many things that can go wrong... Databases are not meant to be used for this kind of problems.

But you know who had the same problem for ages... developers and they came out with great tools like cvs, svn,... and git.

I picked git because it has off-line commits which allow you to work in any situation and that's really important in my opinion, you don't want to connect back to your network just to commit an issue.

So now, the problem of versioning is solved since git is meant to do that and to perform easy merge (or at least tell you when a merge doesn't work properly).

Now, some other advantages of git (shared with some other versioning tools):
  • more than one master repositories... you can have one for storage, one for backup, one for reporting and one for review for example. You can then push your changes to the storage's one and you can push some changes you want to be reviewed to the review's one. Same for the reporting.
  • you can have hooks. Basically you can be notified when something happens on your repository... so for example, you can have notifications on your review repositories that send an IRC message for the tech review and a jabber message for the business review
  • you can have hooks (again.), imagine you have a tool that built the report from your issues, you can get this report built every time you push to the reporting repositories.
  • you can have hooks (again..), you can run `aspell` on every issue when they are committed
  • you can have hooks (again...), imagine you create in your commit a file named 2crack and push it to a specific repository... a hook takes the file, (create a branch), and run your password cracking tools on it
  • you can have branches, you can create automatic script that will put their results in a git branch so you can review them and merge them if needed in your working repository. For example, you can run skipfish (with a 2skipfish file for example) through a hook and get its result in a skipfish branches then merge it back.

However, this method needs a way to store all information as text, hopefully languages like Mardown can be used they're simple and easy (compare to Latex)...

I hope these details give you a better understanding on why I'm so unhappy to use Word to write Security reports.

Report should be seen as a process not just a result...   ;)

Tuesday, 29 May 2012

On certifications...

As you probably guessed it if you read this post, I'm not a big fan of certifications... In this post, I will provide more details on why...

As a disclaimer, I didn't try to get any of this exams, I however read some of the courses ...

There is a lot of certifications in the security market: CISSP, CEH, ISO27001 lead auditor,   ISO27001 lead implementer, GIAC, CREST, ... When I think about that, I have a mental image of a cow (with Security written on it) getting milked by people in suits (all this people have one of this certification's name in their back) ... It's basically what the current market of certifications is in my opinion...

As an opener, this is a little story someone told me: one of this certifications' shop was offering a new certification for free and without any work or exams to some people already certified by this shop but for another certification (selected for their experience obviously). This was just done to bootstrap the new certifications and give it some value ("X people are already B.L.A.H certified, what are you waiting for...")

First, the CISSP, I think the CISSP is a good certification if you want to prove that you have a basic understanding of security. It's kind of like when you want to go kayaking you need a paper saying that you can swim 50 meters... it doesn't really show that you will be able to swim in open water or in strong current, but it shows that you won't drown in less than a minute. The most annoying thing about CISSP is people putting it in their name on LinkedIn or in their emails signature, it's not a PHD or something you need a lot of work to get so please remove it from your name. Some people are even bragging "I know security, I'm a CISSP" to people who really understand security... It just doesn't work that way, are you going to tell a professional swimmer that you have your "I can swim 50 meters" certificate? 

I have mix-feeling about CSSLP, it has been created by ISC2 (like the CISSP) but the content actually seems better and more practical. Even if their marketing is terrible "you will be seen as a leader in your organization":

I quickly read the course book, and I think it's a good introduction to SDLC. However it's not really known or used.

C.E.H. has probably the worst value in my opinion, not that the certification is completely bad (I had some inside on their exam), but because of a lot of people passing them and thinking they know everything about security/hacking because they run msf few times and exploit a SQLi using SQLmap... and let face it the name is intrinsically wrong. There is however some really good people who started with this certification and manage to get really good. If you're CEH, see this certification as a starting point and a quick introduction to security, all the work is still to be done.

In Australia, people started talking about CREST one or two years ago. CREST seems above other technical certifications and is really hard to get. The exam is hard and you don't have internet access (which seems a bit unreasonable)... So you need to have everything on you and be ready with even old exploits in case you see really old systems. There are 2 reasons for the no internet access: you won't be able to leak their exams and you won't get help. That seems fair but is really far from the pentest reality. There is some weird thing about CREST that I however still don't get, if you look at the Member Companies,  and scroll down to "Thales UK Ltd" or "Security Alliance - Paladion / Plynt", you can see that they are listed without any "Certified Application Test Consultants", "Certified Infrastructure Test Consultants" or "CRT Qualified Consultants". I think this is confusing and may lead to mistakes for someone doing a (too) quick check on the website. Finally, the format of the certification is not really easy for small pentest boutiques and it's more likely to help 30+ people pentest shops than small businesses as skills differentiators.

I won't even talk about /ISO-2700\d/ certification since they have nothing to do with pentesting. I guess it's just another world...

Now let's kill one pro-certification argument...
Pro-certification people often argue that you won't get surgery from a non-certified doctor, or electricity from a non certified electrician. This argument is fundamentally wrong. Firstly, being a doctor requires far more than a one day exam... it's a lot of studying and practicing, nothing the security industry can relate to. Secondly, certified electricians are not used because they are better, they are used because insurance companies force people to... I'm pretty sure people with a real passion for electronic and renovation but not certified will do a better work than most of the certified electrician.

Now what is the solution ? No certification, probably not... the security industry is a market for lemons. I think certifications are just something you have or don't have and just mean that you spend one week working on an exam. Nothing more, nothing less... It's just one more line in your resume. Certifications should be seen as the starting point of something and not a goal. However as said before, I often see them as a negative point (especially for people collecting them), since they are currently used to show your knowledge and not just to show your interest/passion.

As a pentester, if I had one week to skill up, I will probably go for:
  • a training at a security conference;
  • a security conference (for talks and to meet people);
  • a week working on a security project;
  • a programming course.
 You will probably get far more value out of these, however you won't be able to hang these in your living room.

Saturday, 19 May 2012

Reporting automation

I'm a big fan of automation... computers have been created to process automatic/annoying tasks...

One of my favorite subject that I still didn't crack is reporting automation. Building reports automatically is in my opinion really important for the following reasons:
  •  remediation is often the same for a given findings and can automatically be improved (like putting PHP function for a directory traversal on a PHP website instead of just generic prevention for directory traversal)
  •  you can prevent some usual mistakes: like bad risk rating of XSS if the HttpOnly flag is set,
  •  you can create pretty graphs,
  •  you can keep statistics on your findings,
  •  and mostly, you can save time.

So far, I found 5 solutions and can't make my mind on which one is the best:

  •  Latex (known as the masochistic solution). Using Latex to generate a PDF, it's probably the solution that will create the prettiest reports but is also the most painful one... and it's not really easy for people coming on board.
  •  Markdown -> HTML -> PDF using Webkit, this is what I'm currently using for Pentesterlab's courses. It looks pretty nice and pretty much everything is done by CSS (ie: you can get a web designer to work on it) so you can get the pretty reports.
  •  Markdown -> HTML -> PDF using Prince, same as above but you have to pay for Prince. However I think the result will be a bit better (Prince's export to PDF is better than Webkit)
  •  Open Office automation using java/python. Unfortunately, I have really bad memories with Open Office and sometimes the result on Word is not perfect
  •  Word automation using C#. Probably the best solution since you have a real Word document at the end but I don't really like the tool chain and don't want to spend few days/weeks in Visual Studio/Windows.
Even if they seem more annoying, the first three solutions have a really big advantage in my opinion: you can easily keep tracks of changes using git/svn/... since you don't handle binary files. However, you need to use aspell instead of Word/Open Office spell check.

Any thoughts/ideas on this?

Monday, 14 May 2012

Hiring pentesters... from the other side

Someone suggested that I should write from the other side of the hiring process... and I think it's a pretty cool idea :)

Just as a side note before starting, I was really interested and surprised by the Hungry Academy. If someone wants to start something similar, I will be really happy to help ;)

Now back to our subject...

Every time I run an interview, there is the moment when you ask people if they have any questions...
Maybe it's because interviewees know really well the company (meh...), talked with someone else before, or just don't want to give a bad impression, but I'm often surprised by the (lack) of (interesting/important) questions.

In the current market (especially in Australia), good pentesters can now choose who they work for (compared to few years ago)... IT security is currently starving for (good) pentesters...

These are some of the questions that in my opinion are really important and that most interviewees should ask:
  • how many engagements have you rejected in the last six months? And why did you reject them? 
  • how long is the average engagement? What is the longest? What is the shortest?
  • how much research time will be provided? will this research time be included in a written agreement?
  • OS running? (I'm old now and have really bad habits: Linux + wmii)
  • How many/Which conferences are allowed?
  • How many/What training can be done/followed? Do you run internal trainings?
  • Main programming language used for internal tools? and why?
  • What do you use for reporting? (I used to write reports with OpenOffice...)
  • What's your biggest weakness? (for one time this question can be asked to interviewers)
  • What's your biggest strength? (for one time this question can be asked to interviewers)
Some of these questions may seem silly, but as you get older you don't want to work in some conditions...

As a second site note, I started working on another project: PNTSTR... an easy way to run the first interview round without wasting too much time.