SoftwareSecurity2013/Group 4/Code Scanning Reflection

RATS

For the security requirements that we were looking in to, the tool RATS is far too limited for finding escaping errors in SQL output. This is because RATS does not walk through the code to simulate execution, so they cannot check if there is a function called somewhere that escapes the SQL input.

RATS may be convenient for quick scanning your code on big mistakes, but is to limited to really tell whether your code is safe. We would not rely on its test.

(Erik: I agree with your analysis here, except that I can imagine that a tool lile RATS could do a better job in detecting unsafe SQL calls. I can imagine that a simple grep-like tool such as RATS could some unsafe constructs, eg uses of mysql_query, or flag the absence of any of the PHP functions to do parameterised queries.)

Fortify

Fortify seems to do a much better job on checking your code, especially the analysis evidence screen is very convenient for tracing steps that result in warnings. It becomes easy that way to verify if the warning is correct. Fortify gives the impression that it really walks through the code, but it results in more false positives, so you will have to verify them yourself to make sure that the warnings are correct. The tool is very extensive, and is reasonable easy to work with, but with the requirements we had to look in to, we did not learn anything new ourselves. It is definitly not like the program can spot more errors than we can on our specific security requirements. With a quick recon of the program, we suspect that on other requirements, Fortify may spot some errors that we would not see directly. Further does Fortify contain a nice knowledge database that provides understandable context information on the security warnings, which can be a good addition to users that do not know all specifics.