SoftwareSecurity2014/Group 8/Reflection

Difficulties encountered

Difficulties we encountered are mostly Wordpress related:

• The amount of files in Wordpress. We were not really prepared for the amounts of code that would need to be checked for a manual and automatic code review. The automatic code review with Fortify gave us a lot of vulnerabilities for which we needed to check whether they really were vulnerabilities, or just false positives. That is why we sampled these vulnerabilities and only looked at the high risk vulnerabilities, which was a good way to reduce the work-load. For the manual code review we only looked at files that were likely to be important for our security requirement, we used sampling here because we could not look at all the code. The vastness of Wordpress was therefore a difficulty, because it is hard to make a definite statement about all of Wordpress when you have only looked at parts of it, but this difficulty was reduced by sampling and prioritizing.

• Wordpress has a lot of spaghetti code[1]. The code is not really structured, which made the manual code review more difficult. A solution for this would be to have clear, structured code, but this is up to the Wordpress developers.

• Wordpress employs a filter system that allows developers to hook into the native Wordpress functionality. This fosters a lot of flexibility for adding or customizing Wordpress functionality, but this also makes it hard to dig through the code.

Reflection on ASVS

The ASVS is very clear and well structured, but could be more complete in our opinion. For our verification requirement, there are pretty broad descriptions of the subrequirements. If the requirements were more specific, it would be more straight-forward to do a security review, as in this case the requirements were sometimes left open for interpretation. On the other side, the ASVS being not too specific allows it to be used in all kinds of environments and for all kinds of programming languages. During the assignment, we interpreted the ASVS requirements in a strict sense.

Reflection on splitting the work

There are several advantages and disadvantages for splitting the validation work as was carried out in this course. We were encouraged to pick one or a pair of security requirements to focus on, while neglecting the others. On one hand, it is useful to know which security requirement you are focusing on. There is no need to worry about cryptography when you are looking at input validation, for example. On the other hand, you won't get a very complete picture of the security requirements if you are focusing on one specific requirement. The code has to be evaluated anyway, so it reduces the workload if you would have to check the code several times for all requirements instead of several times for each separate requirement.

However, the risk of overlooking vulnerabilities is more present when checking the code without a specific focus compared to checking with a security requirement as focus. Therefore we conclude that splitting the security requirements in this way is useful after all, since there are enough students to focus on a specific part.

Facilitating a security assessment

In order to facilitate a security assessment, developers should write clear, consistent code. For example, when looking at the verification requirement that we checked for wordpress, which was input validation, developers should be consistent in validating input, in logging errors when input validation fails, in sanitizing input. The same validation controls should be used throughout the code for the same type of data, this makes it a lot easier to spot security vulnerabilities. This goes for other verification requirements as well. The code needs to be clear so that someone assessing the code for security has a good overview of the workings of the code.

After having experienced security from the side of a reviewer, we would definitely change the way we develop web applications in the future. Not only do we know more about good practices of ensuring security in a web applications, we also understand the need to develop these applications in a way that makes vulnerabilities easier to spot.

Overall reflection

Working with tools like Fortify to check the security requirements stated in the ASVS, was a nice experience. Fortify seems to be a very efficient tool to see function traces, how data flows through various functions. In this project however, we experienced a lot of repetitive work. Validating pieces of code and writing the security verdicts for the subrequirements quickly became a repetitive task.

By doing this assignment, we gained a better insight in the functionality of Wordpress. Since we saw a lot of the code and how the functions of Wordpress are handling user input, a basic knowledge of this was obtained. When, in the future, we are given a new assignment which is similar to this one, we will have a better understanding of the process. We will know where to start and what to look for. We found some nice resources regarding data and input validation, which can be found on the case study page.

Furthermore, we highlight that communication and documentation is extremely important in this assignment. When working with five group members looking at the same security requirement and the same program, it is necessary to have a clear task division and document the work that is performed. The Wiki was a nice platform to use for this. Although some of the group members never worked with the Wiki before, it was easy to learn and use the basics. Because of the possibilities to work separately on the same page and undo changes using the history, it was easy to manage the content. A Wiki error occurred just once: a part of our work was lost and could not be recovered via the history. The cause of this loss is still unknown.

In our opinion, this assignment was educational and a useful addition to the aspects of Computer Security that we learn in the Kerckhoffs master.