SoftwareSecurity2014/Group 8/Code Scanning Reflection

Fortify

This static code analyzer seems at first sight a very useful tool to find vulnerabilities and potentially dangerous pieces of code. This tool scans the program code and sorts the found vulnerabilities in four types of severity:

• critical
• high
• medium
• low (default)
• (all)

Within these severities, the encountered vulnerabilities can be grouped by various options, for example: 'Category', 'OWASP Top 10 2007', 'SANS Top 25 2009'. This is a very useful option if you are interested in a specific category of vulnerabilities. Scanning Wordpress resulted in more than 4000 reported vulnerabilities, of which 1500+ where critical, according to Fortify. By inspecting these critical vulnerabilities, most of them turned out to be false positives. (Erik: It would be good to give a rough indication how many you think you looked at.) This is an important downside of Fortify: it reports a lot of false positives, making it hard to filter real vulnerabilities of this.

A very useful option in Fortify is 'Analysis Evidence'. This option can be used to show the paths that are walked to get to the vulnerability, which is useful to trace the origin of the user input.

Fortify also supports configurable rules, which can be used to reduce the number of false positives by specifying special functions that in fact sanitize or secure certain input or operations, so that Fortify that inputs that have gone through such a function are actually good to use an no longer a threat. (Erik: But you did not try to do that for the custom sanitisation functions, right? Not a criticism, figuring otu how all that works might be very tricky, but I'm just curious) It can also be used to increase the number of true positives by specifying additional vulnerable functions.

RATS

We knew RATS was a pretty simple tool to start with. It basically scans for the usage of certain functions. In the case of PHP it scans for 55 functions (but can be extended by altering a single XML file), which are given below. When used in the wrong way or when exposed to outsiders (for example due to unsanitized parameters being passed to these functions), these functions may indeed pose a threat to the security of web applications.

basename   file            gzgets           is_writeable     posix_mkfifo
bzopen     filegroup       gzgetss          leak             posix_ttyname
bzread     fileowner       gzopen           link             read
chmod      fileperms       gzread           lstat            readfile
chown      fopen           highlight_file   mail             rename
chroot     fscanf          is_dir           mkdir            rmdir
dirname    fsockopen       is_executable    opendir          show_source
eval       getallheaders   is_file          passthru         stat
exec       getenv          is_link          pfsockopen       sysmlink
fgets      gzfile          is_readable      popen            system
fgetss     gzgetc          is_writable      posix_getlogin   unlink

(Erik: Btw, this list suggests that RATS is pretty C/Linux biased)

The tool is able to identify 3 levels of severities (low, medium and high), which allows an inspector to prioritize his job. It is also able to export its results in both HTML and XML formats, which can be interpreted by other software easily. Results are grouped together by the name of the function that was found and it will also give a (short) description of the problem(s) that may exist in using the function. Every occurrence of the usage of a function is specified by the file name and line number on which it occurs.

Due to the fact that it simply scans for usage of the function and doesn't do any intelligent stuff, the latter still has to be done by a human. Results only contain the file name and the line number, so the inspector has to look up the file himself and look for the right place before he or she's is able to assess the secure usage of the function. The fact that in only checks for the usage of certain functions will generally results in quite a big number of false positives.

Due to the simplicity of the tool it is able to scan a lot of code in a really short amount of time, which is of course a good thing. We think RATS will scale up quite well when the amount of files increases or when files get bigger.

What was learned from the tools

By looking at the vulnerabilities found by Fortify, we learned that Wordpress employs user input validation in a lot of places. When looking at the number of possible vulnerabilities that Fortify found, it becomes apparant how important this user input validation is. There are a lot of places where this validation can go wrong and if it goes wrong in one place then that is enough to make the system vulnerable to SQL injections, command injections, cross-site scripting attacks and so on. So one thing we learned is the importance of sound input validation, luckily Wordpress so far seems to do a good job at this.

Our verdict on RATS is that it may give a quick glance of the files that are most in need of manual inspection and that it is quite easy and fast to use, but that it lacks the intelligence to do a real good job at giving advice because it doesn't check the semantics of the program. In the case of input validation, RATS didn't contribute to finding security vulnerabilities in the Wordpress source.

In general the tools taught us to distrust all code where there is user input: whenever it is sent to a web server (XSS), used to open processes (Command Injection), used to create queries(SQL Injection), used to open files (Path Manipulation), and more. In all cases, user input should be sanitized.