SoftwareSecurity2013/Group 3/Code Scanning Reflection

Fortify

We were quite amazed by the functionality of Fortify. It is quite good in finding possible problems. We expected a tool that just did some static analysis on little blocks of code. However Fortify analyses the code as a whole and finds problems spanned over several modules and files. (Erik: Note that this is why Fortify is slow and eats huge amounts of memory :-))

One of the functions we liked is that it can provide a graphical stack-trace with a indication of the modules involved:

However we are not so sure about the severity that Fortify applies to issues. For example we found some issues in the Low section that to us look like big security holes. For example (although not related to our requirements) a possible Command Injection is identified. Furthermore Fortify also identified a possibility for mysql errors to leak and gave it a low priority. We've seen some of these problems in the past and know that that some sensitive information can leak due to these problems.

On the other hand it identified 24 Hardcoded Password issues, 2 with Critical severity and 22 with High severity. First of all, all errors appeared to be false positives caused by internationalization files. And we think that these errors didn't have to be that critical of high but could be Medium.

RATS

We found RATS to be a rather underwhelming tool. It really is nothing more than a glorified grep-like utility for a few vulnerable functions (with a few bells and whistles). Out of all the issues found by RATS the overwhelming majority were completely irrelevant and sometimes so obviously so that it is rather baffling. For instance, the simple fact that RATS does not think "@foo()" is a function call (it is!) is a clear indicator of its limits.

RATS may be useful as a quick check for vulnerable functions but that is really all one can expect from it.

RIPS

RIPS tries to identify areas of the code where user input my end up without being properly sanitized. For that, RIPS identify potential dangerous functions (header, echo, redirect, exit, etc.) and reports them.

Unfortunately, it seems that RIPS does not take into consideration the arguments (calls with static string as arguments were reported as dangerous code due to user input) nor the sanitation that was performed to arguments before the functions calls. This means that the report is nothing more than a list of all the calls of those functions identified as dangerous.

It may be useful to identify places were this functions are being called but does not detect real vulnerabilities. At least, we could obtain the same result with a grep. RIPS's interface is user-friendly and due to the fact that it shows the lines of code that are generating such vulnerability all the necessary data is in the report. RIPS makes it also possible to select which kind of vulnerabilities we want to look at a specific moment and categorizes the vulnerabilities in server and client side. RIPS turns out to be a more user-friendly code scanning tool to use, although with different kind of vulnerabilities, has the same usability as RATS.

PHPLint

PHPLint is primarily a tool used during development to check for style, syntax and/or type errors. As such, it does not have a focus on security and is not very useful for our purposes. It goes without saying that finding these types or errors might lead to the finding of security vulnerabilities. However, if you don't write code the way PHPLint wants you to write code (as is common) PHPLint will find lots and lots of false positive. Our verdict would be: use this tool during development and fix every error it yields (because it helps for writing code in general) but do not rely on it for security.