SoftwareSecurity2012/Group 5/Code Scanning Reflection

Haven chosen the subjects of authentication and cryptography, it is hard to find sufficient tools for checking these two properties. In web applications you look for quite different weaknesses than in, for example, security protocols. Replay attacks can be relevant for web applications, but generally one can use a standard protocol solution, for instance TLS/SSL. Other solutions include salting and hashing of passwords, etc. To verify whether these protocols work, a better solution would be to perform a verification of the protocol, by for example using a program such as ProVerif.

Although it is hard to detect whether protocols are implemented correctly, we assume they are using standard implementations and we should not look for vulnerabilities here. Some vulnerabilities, like SQL injection, can be detected, but it is hard to look for errors in authentication requirements, since these are not natively supported by the PHP language (and require more in-depth knowledge of the system at hand).

Also cryptographic faults seem hard to detect to us, although some checks may be automated (for instance, usage of outdated MD5 hashing or flaws in the cryptographic hashing). It is for example possible to make sure that the private key of a public/private key pair does not get published, but how can a code scanner detect whether this is actually happening?

To find flaws in Authentication and Verification we believe that the only way to find all flaws, or even the most flaws, a manual code inspection is the best possibility.

RIPS

When looking at the output that RIPS provides, we can notice that RIPS gives relatively good output, of which the amount you can customize through several predefined security levels. These levels are 1, 2, 3, 4 and debug. When running the code on level 1, RIPS provides 11 warnings, but when running it on level 4, RIPS manages to find 916 warnings. The warnings are presented in a HTML page, which contains syntax highlighting and listing of the vulnerabilities in orderly sections.

Type of feedback

The type of feedback ranges from file inclusions/manipulation to SQL injection and seems to be quite exhaustive, especially when running the tool on a high level. However the code scanning does result in quite some false positives. When we checked the first level manually, nearly all warnings that RIPS produced were false positives. We have looked into all 11 warnings of level 10 and found 10 false positives. We have also scanned through the entire list of all 916 warnings at level 4, but found mostly duplicate warnings and warnings that did not make any sense. In our quick scan, we did find some (i.e. ten) warnings that appeared useful, but eventually turned out to be false positives or seemed to indicate issues not related to authentication or cryptography. The positive side of the story is that the output is presented in a very ordered fashion, which makes it easy to locate the code which caused the warning and because of the good feedback, you can already see what kind of vulnerability the tools discovers and where the vulnerability can cause harm.

Conclusion

At first sight, the feedback appears to be quite useful, but when examining the actual feedback it appears that nearly all warnings are false positives. This sadly enough also implies that we cannot apply the warnings on any of the security requirements mentioned in the ASVS, especially not ours. It is also our opinion that we might not find any code scanner which can find all vulnerabilities which relate to our security requirement (Authentication/Cryptography).

RATS

We used RATS version 2.3. The tool appears to be very small and simple. It claims to be able to scan source code written in a number of languages and warn about common security programming errors such as buffer overflows and TOCTOU (Time Of Check, Time Of Use) race conditions. RATS supports 3 warning levels and can sort warnings according to estimated severity.

Conclusion

The RATS output is unfortunately completely useless. We have looked into all 7 warnings, but unfortunately all these results were false positives. Furthermore, none of these issues indicated problems related to authentication or cryptography.