Software Security/Group 2/Reflection

??Erik: note that all the reflection seems to be focussed on the tool use. What do you think of the AVSV, the way you used it, any additional manual review you had to do, etc.? Also, you never clearly say how GOOD/USEFUL the tools really were in the end, to find real issue?

Using tools

The use of tools for of code review is really useful because it finds possible risks a lot faster than someone can do by hand. Nevertheless you still need to check all the findings manually because it might be false positives but we think this is not a problem because the tools tell you where to look. Luckily for us we got some information about using those tools during the lectures. This made it more easy for us to understand how such tools work and which kind of tools there are. With this information we were able to place the tools in the category to which they belong. Our experience with using such kind of tools is that the commercial tools have really some advantage over the free tools. The commercial tools give a much more detailed report about the scanning and check the source code at a much higher level (not only a search for keywords). Of course those tools are not very cheap to buy but we think they are worth it, at least, the licensed version of CodeSecure and Fortify are. They really give some detailed output.

Method

By doing the code review we divided it into some steps:

• Install tools
• Run tools over phpBB source code
• Manually look at the results
• Decide which results are interesting
• Create verdict for security requirement

We think this was a good way of working but it is good to note that you really should take some time to get familiar with the tools, otherwise the results might be a little confusing. Once you know how a tool works and presents its results, it is relatively easy to understand.

This way of working really worked well for us. Because you first get familiar with the tool, we were much more able to interpret the results. We think this really helped us by making progress. Jumping immediately to trying to understand the results without looking at the tool would probably have caused some trouble. Since this method worked pretty well we would not change it when we had to do it such kind of project in the future. The ASVS was also quite useful because it tells us what to do at every level. Furthermore it describes which security requirements should be checked. This was also a bottleneck in our project because we found it difficult to find out which security requirements were satisfied according to our subject (Output Encoding/Escaping for SQL).

Planning

To be honest, our planning was not that great. We had a bit of a slow start and had to work harder at the end of the project. Therefore we decided to start with weekly meetings because it seemed to be that this was the only way to guarantee some weekly progress. In those meetings we made a task division for our group members for the next week. Nevertheless there were some weeks in which we did not make that much progress. In those weeks we decided to just add some tasks to the existing ones (so some people had to work harder for a week) because that was the only way to make progress. We also wanted to try the CodeSecure tool (with license) but there were some problems with the license because it was bound to the hardware and the laptop on which it was installed was crashed. Luckily we got a new license in time, so we were able to try this tool also. After all we think our planning was not that great but still we had enough time to finish the assignment. Nevertheless we must admit that we had to do a lot in the last few weeks. So if we had to do something differently in the future, it definitely would be this.