Software Security/Group 7/Code Scanning Reflection

PHP-SAT

Platform-dependent.

It's really experimental. One good side is that it also points out other things like optimization and correctness problems. As for the security bugs, it stuffs them all into a single issue labelled "MCV0000 [One of the parameters does not meet his precondition]", go figure it out yourself. Using die() with a constant string or include() with a string composed of constant values seem to be the only things that trigger this warning, so it seems to give only false positives and nothing useful.

CodeSecure

All that the registration does is make the "Register for a free trial" link disappear, with no download link on the whole site. They are supposed to contact me with the info but this hasn't happened. Bad customer service.

Pixy

It requires too much memory, at least to scan phpBB. Even with 2 GB of memory given to it, it only reports some syntactic problems like multiply defined functions and wrong number of parameters.

I have tried it on another small, private project. It seems that it's not smart enough to do away with some simple cases where it's obvious there's no vulnerability. For example, if there is an INT field in the database and I check the input data with is_numeric(), it still reports a vulnerability. Similarly if I read the value out of an INT column, echoing that value also triggers a warning while it's obvious that an INT column will never contain problematic data. I imagine that even without connecting to the database, there could be a way to configure the tool with the schema of the database, however Pixy doesn't have this kind of configuration option.

The difficulty of using pixy is the fact we can't to give it a directory as input but only one file which compromises the number of vulnerabilities found.

i think that he does not use inclusions and does not give any errors on parameters sent by the GET method (it's difficult to find the result of the processing ) that create false negatives.

The third important point is not writing to a file results, it appeared only in the terminal windows that drive much of a problem because the terminal has a size limit which prevents to see the first mistakes.

the latter problem is most important, the software can not find that very error, and on-line version, the version software is not usable because there are too many errors of inclusion that are detected by the strangely software "duplicate class definition due to include [filename]"

Fortify

Fortify is really to understand why he said that problem as vulnerabilities, there can explain everything and they are understanding the interaction between the various errors found. there are a lot of SQL injection because the PHP used does not use the functions of preparing the application. This creates serious security concerns. we must not forget that phpBB2 is a forum so many request are related to the display menus and posts when the user navigates in the forum this application are much less critical because it's not like a form that could easily be cracked. Here the user should not even have access to the variable used in the query.

Path Manipulation is interesting for our group because we need to understand the access control requirement. for fortify, the Path Manipulation is a risk because you can change the parameter for the function that open the file, with this vulnerabilities you can open the file in write/read and erase the file. ( i think this problem is in the OWASP category V4.3 verify that users can only access data files for which they possess specific authorization. )

Dangerous file inclusion is another problem for our group because you can change runtime variables of the server. it's a problem in the category V4.7 verify that users can only access data for which they possess specific authorization.

Fortify can not handle string constant keys of associative arrays, resulting that if any element of an array is tainted then Fortify considers the whole array tainted. This tainting can spread by giving an arrays element as value to an other arrays element. In this case if the first array had a tainted element, then both array will be considered tainted regardless which element we used as the right side of the assignment. This causes many false positives.

Conclusion

	PHP-SAT	Pixy	Fortify
Availability	Free	Free	Commercial
Implementation language	Stratego/XT	Java	Java
Installation	Hopeless on Windows; medium to easy on Linux	Easy	Easy
Usage	Easy	Easy	Easy
Completeness of output (for this project)	Scarce	Scarce	Quite complete
Verbosity / clarity of output	Quite clear for optimization problems Uselessly ambiguous for security problems	Quite clear	Quite clear and detailed
Rate of false positives	High (at least for the security part)	Not enough data to decide	Medium
Usefulness for our project	None	None	Medium