SoftwareSecurity2014/Group 1/Code Scanning

Fortify

We ran Fortify with a number of different options and settings. Below we list the different ways and what output they produced. Mainly we focused on the difference between quick scan mode and 'normal' scan mode. There are Rulepacks that one can configure by language, but enabling or disabling irrelevant ones has little influence on the results. This is not surprising as there are no C++ issues, for example, to find. Please look at our reflection page for an analysis of these results.

Scanning options

We ran Fortify in three different ways. The first two were performed from the GUI using standard scanning mode for security issues only and then using quick scanning mode. Normal mode eventually yields an error about memory, which is strange considering memory usage as reported by the OS (Ubuntu, Mac OS) never gets anywhere near problematic levels. In order to try and work around this we ran the analyzer from the command line using manual Java options to allocate more memory.

The command line scan was performed like so:

Step 1: Clean

sourceanalyzer -b FLUX -clean

Step 2: Translation/Build

sourceanalyzer -b FLUX -Xmx1280M -debug -logfile trans.log .

Step 3: Analysis - Only for PHP

sourceanalyzer -b FLUX -Xmx1280M -debug -logfile scan.log -no-default-rules -rules ~/HP_Fortify_SCA_and_Apps_3.80/Core/config/rules/core_php.bin -verbose -scan -f FLUX.fpr

Scanning results

The following table lists the critical issues that were found. For the purposes of this evaluation we are enumerating only these. Fortify reports another handful of 'high' level issues and many lower level ones. We suspected most of them are false positives so we allotted our time to the important ones.

Critical issues

Category	Normal	Quick	CLI
Cross-Site Scripting: Persistent	98	0	98
Cross-Site Scripting: Reflected	70	74	70
Dangerous File Inclusion	4	4	4
Dangerous Function	2	2	2
Dynamic Code Evaluation: Code injection	2	2	2
Open Redirect	3	4	3
Password Management: Hardcoded Password	2	0	0
Password Management: Password in HTML Form	2	2	2
Privacy Violation	6	6	6
SQL Injection	55	80	55
System Information Leak: External	1	1	1
Total	245	175	243

Two things stand out from these results. First, it is interesting that in quick scanning mode some categories contain more issues than with the supposedly more thorough scan, although overall the normal scan spots more issues. (Erik: Indeed, that is weird. The only explanation I can think up that in quick mode Fortify gives up on taint analysis at some stage, at the expense of more false positives.)We note that Fortify complains about too much code complexity in three files when run in the regular scanning mode: db_update.php, moderate.php and profile.php. As a result, it refuses to scan them.

In contrast, when the quick scan mode is used, the errors about two of the three files disappear. Since these files are scanned in this mode, their potential issues can be found, which could explain the increase in certain issues for the quick scan. Even in quick scanning mode, db_update.php appears to be too much for Fortify to scan. Still, this does not explain why Fortify's quick scan can cope better with more complex files than with regular ones.

Second, even though the normal GUI scan appears to terminate in an error (not enough memory) and the command line scan completes successfully, the former reports more issues. Just to make sure, we also ran the quick scan from the command line. The results are the same as the GUI version, however the analysis from the command line is much faster. A normal scan takes 4m 18s and a quick only 2m 48s. On the other hand the GUI normal scan takes 25m 10s (on a 2013 TU/e laptop).

RATS

Scanning options

The verbosity level of RATS's output can be configured. We used the highest level to get as many results as possible. Also, we did not use the language ("-l php") option because it was producing too many errors and automatic detection works well.

After various tests, the most meaningful results were produced by the following command:

rats -w 3 . --html > rats_results.html

Scanning results

The RATS scan resulted in the detection of 58 instances of 15 issues. The tool issued a few warnings about unsanitized data passed to functions, a few TOCTOU (Time Of Check, Time Of Use) problems and many buffer-related issues.

A sample of the issues is given below:

Severity: High
Issue: mail
Arguments 1, 2, 4 and 5 of this function may be passed to an external program. (Usually sendmail). Under Windows, they will be passed to a remote email server. If these values are derived from user input, make sure they are properly formatted and contain no unexpected characters or extra data. 

Severity: High
Issue: fopen
Argument 1 to this function call should be checked to ensure that it does not come from an untrusted source without first verifying that it contains nothing dangerous.

Severity: Medium
Issue: fsockopen
Argument 1 to this function call should be checked to ensure that it does not come from an untrusted source without first verifying that it contains nothing dangerous.

Severity: Medium
Issue: is_readable
A potential TOCTOU (Time Of Check, Time Of Use) vulnerability exists. This is the first line where a check has occured. The following line(s) contain uses that may match up with this check: 44 (chmod)

Severity: Low
Issue: fixed size local buffer
A potential race condition vulnerability exists here. Normally a call to this function is vulnerable only when a match check precedes it. No check was detected, however one could still exist that could not be detected.

PHPLint

Scanning options

PHPLint can be run multiple ways, including as a feature on the PHP command line or using an online checker [1]. It's designed primarily to convert PHP from a weak-typed language into a strong-typed one. Individual files can be checked as an aid during development using different packages and modules.

Scanning results

118:            $tpl_temp = count($required_fields);

                $tpl_temp = count($required_fields);
                                                    \_ HERE
==== 118: ERROR: cannot assign a value of type int to a variable of type string
119:            foreach ($required_fields as $elem_orig => $elem_trans)
120:            {
121:                    echo "\t\t\"".$elem_orig.'": "'.addslashes(str_replace(' ', ' ', $elem_trans));

                        echo "\t\t\"".$elem_orig.'": "'.addslashes(str_replace(' ', ' ', $elem_trans));
                                                 \_ HERE
==== 121: ERROR: `... . EXPR': found EXPR of type mixed, expected a string

                        echo "\t\t\"".$elem_orig.'": "'.addslashes(str_replace(' ', ' ', $elem_trans));
                                                                                                           \_ HERE
==== 121: Warning: calling `addslashes()' declared in modules/standard:420, argument no. 1: found type `mixed', required type `string'
122:                    if (--$tpl_temp) echo "\",\n";

                        if (--$tpl_temp) echo "\",\n";
                                        \_ HERE
==== 122: Warning: invalid type int assigned to string

                        if (--$tpl_temp) echo "\",\n";
                                        \_ HERE
==== 122: ERROR: `if(EXPR)': expected expression of the type boolean, but found an integer value. Remember that 0 evaluates to FALSE, and any other integer value evaluates to TRUE.

PHP Code Sniffer

Scanning options

PHP Code Sniffer [2] does not have many scanning options. It only requires a coding standard and the root of the directory where the source code is located. Furthermore, the reporting output is simply sent to stdout. The exact command we used was:

phpcs --standard=PHPCS . >> report

Scanning results

Although a small program, the output from this tool was ridiculously large. It reported 5122 errors and 462 warnings after scanning only 1419 lines. That is roughly 4 problems per line of source code(Erik: :-) that's a bit too many to be useful...) .