SoftwareSecurity2012/Group 3/Code Scanning Reflection

Discussion of RIPS

In this section we discuss RIPS and comment on the results of the tool. For more details on the scanning results, see the code scanning results page.

General

RIPS is somewhat harder to install than most of the other scanning tools. It requires a running webserver on which the scanner is run. Luckily, we already had an apache server running which allowed us to setup RIPS in a simple manner. RIPS allows to scan for many different kinds of Server-side or client-side vulnerabilities. For us, scanning on cross-site scripting vulnerabilities is the most useful. RIPS also allows us to load an entire folder at once, in contrast to RATS for example which requires us to specify each file separately. The scanning time of RIPS is slower than RATS, but it does check for a bigger amount of problems. The scanning time is still very reasonably at about 20 seconds (for XSS only).

Feedback from the tool

RIPS allows a user to specify 1 of 5 different verbosity levels, which determine how many warnings and which kind of warnings the tool will produce. We used verbosity level 3, which gave us about 3800 XSS warnings. RIPS also shows statistics about the results, showing how many warnings, what kind, the scanned files, etc. It also has the capability to show, for each kind of call how many times it appears in the code. This is a nice addition to learn abit more about how the code is setup.

Conclusions

Overall we think RIPS will be a useful tool for us. It shows warnings that are interesting for our requirements and has a wide range of different options. These options can be used to tailor the results of the tool to your specific needs, in our case XSS warnings.

Discussion of RATS

In this section we discuss our experiences with RATS and the usefulness of the tool.

General

Installation of RATS is moderately difficult (on Linux). On windows we were unable to get it running, due to the fact that it needs the Expat library. This library can be downloaded for windows and installed, however RATS will still complain that it is not present. On Linux it is easier to install RATS following the guidelines provided on the site. The command line interface of the tool has some well thought out functions, however it requires you to specify every file you want to have scanned separately. This can be a real problem when the amount of files is big. For FluxBB the command was already several lines big (see the discussion of the code scanning results of RATS). One of the positive sides of the scanner is that it runs through the entire code very fast, although this is likely a result of the limited amount of checks the tool actually performs.

Feedback from the tool

RATS is a limited code scanner that only checks for a certain set of problems in php files. As a result of this, the amount of warnings the tool gave about the code was also very limited. Most of the warnings were focused on the use of certain php functions (like fopen), which might be unsafe. The word might is important here, since the tool gives a kind of surrounding context which, when true, will imply the use of the function is unsafe. It is however not always the case that this context is true, and this is not checked any further by the tool.

Conclusion

The results we got from the scanner where not very useful for our security requirements. Most of the warnings given by RATS will also be given by different scanning tools, although perhaps a higher warning level is required. For our purposes this tool cannot be considered useful.

Discussion of YASCA

In this section we discuss the tool YASCA and our experience with it.

General

There is no installation procedure of YASCA. After downloading it, you're good to go. YASCA is executed by calling the executable with the path of the source. The execution time of YASCA is short and it is possible to let YASCA make use of other tools like RATS or JLINT as if they where plugins.

Feedback from the tool

The tool gave a little over 100 errors from which almost half where Possible Licensing Restrictions. These errors are not very interesting and should be combined into a single one. The other errors where quite interesting and contained only few false positives.

Erik:but for the XSS warnings that remains to be seen, right?

Group: We use RIPS for the XSS warnings, but from what we can tell the YASCA XSS warnings are false positives because the correct escaping is performed using the htmlspecialchars function.

The feedback also contained the part of the code and more information about why the error was thrown. The tool also gave performance specific errors which contained no false positives.

We've executed YASCA with RATS and JLINT enabled. This produced thousands of errors. We concluded that there was no benefit of running RATS or JLINT as a plugin over running all three separate.

Conclusions

The tool is quick and gives little but quite accurate feedback. The performance specific errors where quite useful.

Discussion of CodeSecure

In this section we discuss the tool CodeSecure.

General

CodeSecure was only downloadable from the official website. Downloading the tool took 2 hours, since it's 103MB and downloads at 10-19Kb/s. To install the tool you should first install a service which is basically a small web-server where the GUI runs. Installing the tool will recommend you to use a 64bit Java Runtime Environment. After installation a license will be requested. We have an academic license which let us use the full functionality with only one restriction: Only projects with <10.000 lines of code where supported. Browsing to http://localhost:8080/ takes you to the GUI which looks very professional. There are a lot of options and the GUI is quite intuitive.

The system requirements are really high; Source

• 64-bits CPU running at 1.86GHz or higher
• 8GB of memory or more (8GB recommended)
• 50GB of available disk space

Feedback from the tool

Directly after trying to use it the following error occurred:

" Scan Aborted The license restriction of 10.000 lines of code was exceeded, as the analysis found 'at least 10.364 lines of code. The scan was aborted."

After removing a lot of comments the number of lines of code indicated by the error increased. So the most logical next step was to count the lines of code. The tool Count Lines of Code ([1]) was used which gave the following output:

> cloc-1.55.exe fluxbb-1.4.8

    164 text files.
    136 unique files.
     25 files ignored.

Count lines of code v 1.55 T=1.0 s (113.0 files/s, 42403.0 lines/s)

Language	files	black	comment	code
PHP	98	4932	3102	20529
CSS	12	2852	676	10179
Javascript	2	24	82	26
HTML	1	0	0	1
Sum:	113	7808	3860	30735

Since the project contains 42403 lines of code we can't use our current CodeSecure license.

Conclusions

With the current license we unfortunately can't make use of the tool. The GUI looks very professional so it's a pity that we weren't able to make use of the tool.