SoftwareSecurity2014/Group 2/Code Scanning Reflection

Code scanning reflection

Reflection on Fortify

At the heart of fortify lies the static code analyzer. It is a command line program intended to be integrated in a project's build toolchain, for example in nightly builds. Then there is the audit workbench, which is a GUI to manage the issues found by the analyzer. The audit workbench can also be used to explicitly call the analyzer, and provides wizard dialogs to construct long commands. This is useful for complicated projects where you want to exclude certain files or rule sets.

The analyzer works in two stages. The first stage translates source files to an intermediate representation, stored on disk. The second stage matches rules against the intermediate representation. This way, many different programming languages are supported. There exist rule sets that can be used with all programming languages, and rule sets that look for problems specific to a language.

The two-stage approach requires it to run the translation phase again when the source files change.

The analysis stage runs 6 different analyzers. Rules are always written specifically for one of them. These analyzers exist:

• Dataflow: global taint propagation analysis.
• Control flow: checks whether operations are executed in correct order. Detects things like time of check/time of use, use of unitialized variables, etc.
• Semantic: detects dangerous use of API functions. For example use of deprecated or unsafe functions like printf or gets.
• Structural: detects violations of secure coding practices by analyzing the global program structure.
• Configuration: checks for sane values in configuration files, like timeouts of user sessions.
• Buffer: limited inter-procedural buffer overflow analysis.

Fortify comes with excellent and extensive documentation. The sheer amount is overwhelming at first, but once you have a rough overview of where to look for which topic, it answers most questions. The documentation includes guides on how to use the various tools, but also references for things like custom rules and even the type system that the static analyzer uses internally.

Contrary to the statement in the lectures that code scanners generally favour false negatives over false positives, Fortify gives upon the first preliminary scan (the complete core, no tailored settings) 1000+ false positives. Fortify claims that it found a lot of hard-coded passwords in the files. The problem is that OwnCloud is quite verbose, which means that there are a lot of strings to be found (error messages, strings in the gui) that contain the word "password". Every time Fortify encounters the word "password" somewhere, for example in a string, it flags that piece of code as a hard-coded password.

Because of this large number of false positives, the code reviewers may tend to ignore most password management critical issues.

For all hits, Fortify provides background information about the general issue and some recommendations on how to fix the issue. This information is not specifically tailored to the scanned code, unlike recommendations from RIPS and PREfast, which would include specific changes to the code. Fortify assumes that users are proficient enough in the programming language to solve the issues themselves given enough background-information.

Fortify and un-escaped output

Wondering why Fortify didn't detect any XSS vulnerabilites, we removed all output escaping and scanned the project again. Removing the output escaping was fairly easy, because ownCloud uses two functions to produce output: p() and print_unescaped() , where the p()-function escapes output before printing.

We removed escaping in the p()-function and scanned ownCloud again. However, Fortify didn't find any new XSS or other output escaping leaks. Even including an explicit XSS reflection bug by printing a POST-value unescaped on a page didn't trigger an XSS warning.

It turned out that firstly, we didn't scan the whole source code of ownCloud. Upon closer inspection of the project's structure we saw that only core functionality is in the directory we scanned. A scan of all of the source code revealed some XSS issues. (Erik: But do the comments in the two preceding paragraphs (which suggests that Fortify is really bad at spotting XSS) still apply when you scan the whole code? Or is the behaviour you report in these paragraphs only due to the fact that this code is not analysed?) Secondly, as described above, Fortify applies rules to the intermediate representation, which must be explicitly rebuilt when the source code has changed.

Reflection on RIPS

RIPS is a static source code analyser for vulnerabilities in PHP scripts (http://rips-scanner.sourceforge.net/)

Although RIPS found a large number of positives (5500+), this information was of little value for the project. RIPS will flag a lot of 'normal' code based on the presence of one of more relatively innocent keywords. This leads to a large amount of false positives. We have the idea that RIPS has not found anything of interest, so we don't think that it is a good tool to use for code evaluation.

Reflection on RATS

Rats, the Rough Auditing Tool for Security is a rather simple tool. It only does some lexical scanning, there is no semantic analysis involved. Its vulnerability database is a list of keywords that are looked for during a scan.

An example entry of the database looks like:

  <Vulnerability>
    <Name>mail</Name>
    <Info>
      <Severity>High</Severity>
      <Description>
        Arguments 1, 2, 4 and 5 of this function may be passed to an external 
        program. (Usually sendmail). Under Windows, they will be passed to a
        remote email server. If these values are derived from user input, make
        sure they are properly formatted and contain no unexpected characters or
        extra data.
      </Description>
    </Info>
  </Vulnerability>

Rats lexically matches each <name>-tag against given source code. When it finds a match, it prints the Severity-tag and the Description-tag as advice. This generates usually lots of false positives: When RATS scans source code, it complains about every potentially dangerous function that is listed in the vulnerability database, even if the function is used in a safe way.

It is on the other hand easily possible to extend the vulnerability database by adding entries to an XML file.

It isn't possible to suppress instances of warnings at places you consider safe, so RATS always outputs warnings for all occurrences of the functions in the database. For example, when ran on ownCloud, RATS produces about 3400 lines of warnings, which is obviously too much to analyse. Additional processing is required to use the output in the programmer's daily routine.