SoftwareSecurity2013/Group 3/Reflection

What difficulties did you encounter, either with the code or the ASVS? Can you think of ways to reduce or avoid these difficulties?

In the beginning we had some difficulties understanding the actual definition of the requirements of the ASVS. Some of these requirements are vague and we didn't understand what exactly was meant. For example if you look at V2.5: Verify that all authentication controls (including libraries that call external authentication services) have a centralized implementation. What exactly are authentication controls?

Also, requirements like V2.1 (Verify that all pages and resources require authentication except those specifically intended to be public) are vague in the context of the application we are reviewing. It is unclear what is intended to be public and what is not. So you need documentation about this. In a lot of cases you can ask this from the developers, so it is less of a problem.

Could the ASVS be clearer, more complete, or structured in a better way?

Yes. Maybe giving solid and clear definitions for the definitions would be better. Even giving some examples on what is and what is not a failure of the requirements would help. Note that the ASVS does provide a glossary with some terms. However not all terms are covered.

Is splitting up the work as we did,with different groups looking at different security requirement, a sensible way to split the work? Or are there more practical ways?

We think that it is reasonably practical. There may be some overlap between requirements and the downside is that everybody has to look at all the code. However splitting the work up into functions or files to review for all the requirements may be harder because there can be lots of dependencies on other files.

What could or should developers do to facilitate a security assessment? Do you think that experiencing security from the perspective of a reviewer rather than a developer has changed the way you would design or implement a web application yourself?

Yes it changed the way I would design a web application. The requirements gave me some new insights about how to develop certain parts of an web application. I think that developers should not only look at the requirements after building the application to review the security of a system but also before starting with development. By doing that they can keep some requirements in mind while designing systems. Futher more developers could use the nomenclature that ASVS uses and document which functions are implemented where. By doing this a reviewer wouldn't have to scan through all the files but could use the documentation to find the relevant parts for each requirement.

Any other interesting, frustating, boring, educational, etc. aspects of the whole experience.

I was amazed by the power of the Fortify analysis tool. I didn't expect it to find such complicated problems. Also the manual review of FluxBB showed some interesting problems that could really be exploitable. Many of these problems despite the power of Fortify weren't detected and only appeared during manual review. On one side this shows the need of manual reviews and on the other hand it shows that there is still a lot of progress to make in static code analysis.