SoftwareSecurity2012/Group 6/Code Scanning Reflection

Comments RIPS

General Comments

The RIPS version we tested was 0.53, which is the most up-to-date version at the moment of writing. It was updated in March 19, 2012 and seems to be current.

RIPS is fairly easy to install as long as a web server and PHP are already installed on your system. Simply unzipping the folder into the home directory and navigating to the folder means the tool is ready to work. The code also needs to be unzipped in the same directory and the path to the code given to RIPS. It permits the user to scan file by file, or an entire directory and subdirectories.

In general, the tool seems to be very easy to install and use, and very helpful. The fact that it is being kept up to date, considering all the vulnerabilities that are being discovered, is significant.

Performance

RIPS only works on PHP applications, and finds a variety of vulnerabilities including code execution, command execution, XSS, header injection, SQL injection, file inclusion, and several more. The output is very detailed, as it shows precisely where in the code the vulnerability occurs by outputting the lines in the code and showing the flow of the code.

Comments RATS

General Comments

RATS is available via the package repository of the Linux distribution Debian. After installation, a "rats --version or "rats --<something else not known to the tool>" throws a segmentation fault. From this point on, it is hard to develop trust in the tool.

The tested version was 2.3, the newest available version. Being copyrighted in 2001 and 2002, it does not seem like this tool is still developed.

The tool supports the languages C, C++, Perl, PHP and Python.

Performance

When fed with the FluxBB PHP-sourcecode, the tool gives various warnings, depending on the warning level. Filtering out only high warnings, the tool detects one use of a possible untrusted source in fopen(). On medium level, a TOCTOU-Warning (Time of Check, Time of Use) is added. Going down to low level adds some possible race conditions to the output. But no matter on which warning level, it never gives information about our topic, Output Encoding/Escaping (SQL). This is probably because attacks of this type were not yet common in 2002, when RATS was still developed.

Summed up, RATS would have been a nice tool if it would still be in development. But unfortunately, this is not the case and the tool suffers because of its age.

Comments PHPLint

General Comments

Installation was straight forward without requiring many dependencies from other packages. The tool was deployed in Ubuntu 11.04.

PHPLint is a syntax validator focusing on code readability. It implements known best programming practices to decide if the code is well-typed. Considering these facts, most of the generated alerts are related to the type variable; if for example, within a function the i variable is declared type integer and at execution end i returns a string, an error is output. On the other hand, warnings are generated when a variable is not initiated; a fatal error might be returned, for instance, due to a valid but old-style syntax of a conditional statement or built-in fuction.

Performance

This tool allows to scan only one PHP file at time. It might be designed this way due to the large amount of returned data, which sometimes can be overwhelming. Unfortunatelty, it does not perform a check on SQL statements, which is of our particular interest. It can be a great aid when developing a large project, where many people are involve, giving them the possibility to determine that all variables are well-typed and initiated as well as checking the correct syntax is used throughout the PHP documents.

Comments Yasca

General Comments

Yasca is, according to its website, a "glorified grep script" that can be extended with other open-source tools for code analysis. The feedback generated by the tool is given in a HTML document, sorted by different kinds of vulnerabilities. Without any extensions, the tool conducts only some simple analysis.

In our case, it detects some Cross-Site Scripting vulnerabilities, statements not conforming to the W3 standard and some suggestions for better performance.

The extensions include mostly tools for analyzing JAVA source code. Among those extensions are also some PHP analysis tools: PHPLint, RATS and Pixy. For the two first ones, we already report on the results in another section.

Performance

For the sake of testing the tool, we only tried the extension Pixy. Pixy is a tool that detects Cross-site scripting and SQL Injection vulnerabilities. In our case, it detected more than 1500 XSS but no SQL Injection vulnerabilities.

In general, the feedback of the tool can help in checking the conformance of a web-application to the OWASP standard, for example regarding XSS vulnerabilities. Concerning the analysis of Output Encoding/Escaping (SQL), however, we have to conclude that the tool did not provide any useful output.

Comments CodeSecure

General Comments

The CodeSecure tool is very useful overall. The user interface is superb and it functions well with many scanning parameters. The feedback we acquire is specific vulnerabilities, classified according to OWASP threats and is well expressed and well represented in graphs of severity and depth. In default mode, the output is reasonable but with more extensive/experimental options we acquire much more output. Also, it is easy to trace back to the source code from the tools and we are offered with the option to use multiple traceback points to the same vulnerable code.

Performance

With regards to SQL output validation, we did not encounter any sql related warnings (at least as part of our ASVS level 1 analysis thus we see limited usage with regards to output validation requirements (V6). Our main security requirement, being V6.4, starts at level 2 so we will aim for a more targeted and in depth analysis with heavier tailoring on the CodeSecure tool. ASVS level 2 relies less on automated tools, thus CodeSecure (or any tool) is not expected to perform exceptionally. In general, the feedback is related to several OWASP vulnerabilities (mainly XSS) so it is more closely related to HTML/javascript related vulnerabilities.

Overall, we see potential in the tool, especially with targeted application and specialization on the data sources (CodeSecure allows us a certain point of control over the I/O data sources that handle data. We may choose to include or exclude some, depending on our analysis, specifically we will focus on SQL-related I/O sources). The max code size (which is limited to 10k lines of code) creates problems to the data flow analysis because we are forced to exclude parts of the code that may be linked. Thus, the analysis may miss certain interconnections e.g. fileA.php invoking fileB.php, since fileB.php may be excluded.