SoftwareSecurity2013/Group 12/Code Scanning Reflection

Describe your impressions about the tools, in capabilities, limitations, etc. Also, did you learn anything about specific security vulnerabilities from using them?

General Approach

The approach that we used in scanning the source code of the fluxBB and looking for vulnerabilities within it was heavily oriented towards the points defined in ASVS. Fortify turned out to be a very powerful tool which generated hundreds of warnings for us. Being able to search, group and categorize according to popular standards also came very handy. However it would not make sense that we looked into each and every warning there trying to correlate it with a verification point in ASVS but indeed the other way around. We looked for supporting evidence (warnings) for ASVS verifications points in the report of Fortify and reported our findings on them respectively. Only in this way we can assess correctly the usefulness of the tools used in this phase.

RIPS

- Installation - Features - Limitation: doesn't support object oriented - Functions interdependences, limited in number in the graph

Installation and usability were quite handy and did not result in any problems. However using RIPS for our code review turned out to be not so useful. As it can be seen from the generated statistics after scanning the whole codebase, the majority of warnings (88/103) are about Cross-Site Scripting. Moreover, the usability of the tool is not very friendly nor flexible since there is no easy way to look through the categories, vulnerabilities and explanations from the tool and keep the track of the analysis despite the fact the tool also gives the options to create a graph with the relation of the functions, a complete list of variables/functions and a limited search engine. In general, this tool is useful only as a support and comparison reference for other more complete tools such as Fortify.

Fortify

The installation of Fortify went smoothly by using the scan wizard to get a .bat script to generate the intermediate language project ready for the analysis in Fortify. Using Fortify, we can conclude that the tool produces lots of warnings for FluxBB 1.5, especially for the categories Access Control and Cross-Site Scripting. By evaluating our set of verifications we can conclude that Fortify seems to be useful for looking at HTTP Security, since some of the verifications can literally be found in the warning that Fortify produces. But when we look at Session Management and Communication we can conclude that Fortify doesn’t produce useful warnings for these categories and manual code inspection is required.

The warnings from Fortify were checked manually, since there was a chance that there were any false positives among them. After inspection of the code it appeared that of the 35 warnings, that belong to that category, 8 of them were actually about the requirement that redirects do not include unvalidated data. All 8 of them were false positives as the data was indeed validated by the code. It is hard to say that there are no false negatives, because Fortify cannot be trusted for 100%. This is why Fortify functions more as a guide for how to look at the source code. We used several databases for Fortify that were able to detect certain flaws, but it was remarkable that there was no database available that looks at the Application Security Verification Levels defined by OWASP(Erik:I guess the ASVS, or indeed the whole practice of doing security evaluations, is still not mainstream enough...) . We were disappointed there was no such database, but it makes sense since the practice of doing security reviews of (web)applications is still not mainstream. The lack of such a database made it hard to map the issues that Fortify detected to the specific requirements formulated by OWASP.

RATS

RATS is a multiplatform source code analyzer for Windows and Linux based systems. In order to execute the tool it is necessary to install the additional library Expat. The tool has a limited functionallity for detection of a variety of vulnerabilities. It triggered warnings focused mainly on buffer overflow and TOCTOU (Time-of-Check Time-of-Use) race condition vulnerabilities. In fact, the results given by the tool were not very useful and also there were only a few number of warnings. The usability is not very friendly since it is a command-line tool but is possible to get a report in html or xml format which it is just a list of potential weaknesses within the code but there is no useful in-depth analysis by the tools and therefore a manual verification has to be done anyway. We tried with different execution options and verbosity levels, but the output was almost the same. The execution line was:

$ rats -w 3 --context --html > report_rats.html

Acording to the options of the tool (--database), it is possible to use additional databases/rules in order to trigger more warnings during the analysis, however no additional sources could be found for this tool.

It appears that none of the RATS warnings are relevant for the security requirements we are looking at, so in our case it is not a very useful program.

PHPLINT

This tool is basically a documentation generator which provides a list of functions within all the source code files. Since it is not a source code analyzer, is not useful to find vulnerabilities within the code but it just a reference for the components of the source code. PHPLint can be used using the command line or using TK graphical library.