SoftwareSecurity2012/Group 1/Reflection

Reflection in general

Rolf

For this project we have almost completely seen the process of verifying the security of an application by different means. Because of this I think it is a very instructive project to do. Although this is off course not the full effort needed for verifying actual programs in a non educational environment. Also for some programs a Level 4 verification will be needed which we didn't had to do.

We have seen which requirements the OWASP prescribes and in which way they should be met. I think it is a good thing that there are some guidelines on how to preform such verifications as I can imagine this can be done in a lot of different ways.

For our project each group had to look at a different security aspect of the application which is a good devision of work load. Although I think that the interpretation of how vulnerable a application is can differ between certain groups. Sometimes you are not sure if an aspect is secure, and hopefully mention that. Maybe it would be a nice idea to have each security aspect covered by two groups to see if differences occur. This can also lead to some discussion after the verification which is a good thing to learn from other people. Also there are some requirements which overlap each other or there other requirements you will see things of when examining your own requirement. The devision of requirements in the OWASP standard is most of the time not divided as well in the application(code).

After reading some wiki pages on the OWASP page I found out that they do not really require the verification rapport to be in a certain format/layout. This can be a good thing so that certain projects can make the verification in their own style and according to their needs. On the other hand this can lead to differences in rapports. Maybe a nice reading material is the OWASP Code Review Guide V2.0. It should provide some more information on how the testing should be done, as well on how to write a report out of the testing results.

The automated tools are not really useful for our topic of verification. The tools do not mention anything about vulnerabilities in the session management. My idea is that the tools are not really testing for issues here. I also think that this is not easy to test for. How can you test if for example all aspect of an application use the default session management system? An option could be with annotating code files; for example indicating which is the default session management systems and indication which aspect of the program use this.

Gijs

I agree with Rolf's reflection and I'd like to add the following remarks.

The project gives a nice insight in how the security of a (nontrivial) piece of software may be analyzed. The OWASP requirements are, in general, a nice set of things to have in your web application.

Some applications may be less concerned with some parts and thus may the work distribution of this project be unfair. On the other side the groups are not of equal size so I suppose it doesn't matter too much.

It is still clear that this project used to be about phpbb because it's name is mentioned on several locations. This is not a big issue in doing the project, but maybe some parts of the exercise do no longer apply or are useful to check in fluxBB.

In the first step, the code scanners yielded very little useful information. It seemed like the code scanners are not really useful, on the other hand this may indicate that there are no (to little) problems with the application. The scanning rules are (very probably) sound and incomplete so this is a natural result.

Paul

I mostly agree with Rolf and Gijs on this subject. In practice the OWASP requirements are very useful as a good checklist to check for common dangerous mistakes. In practice it is not enough to just use the OWASP as a checklist. The complete code needs to be reviewed to be certain that everyhting works as it is supposed to work. The code scanners have potential to be helpful. But most the the time they do not give very useful results. As Gijs remarked this might very well be the case because most of the code is fine. The small mistakes found are more of the semantical nature then of the syntactical nature. Therefore it is not odd that these mistakes were not detected by the code scanners.

Specific for the Security Requirements

Rolf

For some higher level verifications (Level 3 & Level 4) there are some additional things we can check, like for example that generating the cookie is secure. This is not in the scope of the requirements verification. Although I will mention some interesting functions that could be used for this verification. On the OWASP website there is also a list of common session management problems. The requirements/mistakes listed here are more applicable for level 3 and level 4 of a verification.

Interesting functions:

functions.php:check_cookie() --> used for checking the cooking authentication

functions.php:random_key() --> generate a random key used for the $cookie_seed stored in the config file

functions.php:foum_hmac() --> to check if there has been any tampering with the cookie

Gijs

There was quite a big difference in the amount and type of work required.

Requirement 3.5 took me quite a lot of boring work. I had to check every file for the presence of a specific line of code or some other line that made that line unnecessary. This may be typically something that a code scanner can do for you, but then you have to specifically make a tool that does this. (Erik:Couldn't you grep for this line of code? Note that one could consider doing the check in say half of the files, randomly chosen, and if it is correctly done in these files assume it is done everywhere. You may miss something then of course, but it may be the best you can do in limited time. Gijs: We did that but it very often appears inside a case statement or under some conditions. This results into that we have to check everything seperately to see if they are included in any case. )

The concerns about the other requirement I checked, 3.6, were mostly centered in one file (db_update.php). For this file I had to check how some functions worked and figure out the construction with big case statements during different stages of the update process. This was quite specific work.

From one part it is nice to have some variation in your work. On the other hand, if someone only does work in the likes of what I did for 3.5 it gets very boring very quickly. Small mistakes can also slip in during prolonged periods of these kinds of vigilance tasks.

Paul

Some of the requirements are either ambiguous or assume some implementation details. For instance consider the requirement: Normal 0 false false false EN-US X-NONE X-NONE Verify that the session id is changed or cleared on logout. First of all it is assumed there is some clearly defined session identifier. In this situation this is not really the case(Erik: But isn't the cookie set on login effectively a session identifier?) . There is a cookie with stores the properties of the session and there is a HMAC that prevents tampering. After logging out the cook is changed to guest credentials so one might be tempted to consider this requirement as passed. On the other hand the cookie (if someone still has a copy, for instance an attacker) is still functional. Therefore this is a security problem and it makes more sense to consider this a fail (Erik: Good point. Irrespective of the ambiguity in these V3.*, I would make sure to flag such a problem is at least one FAIL somewhere, even if you take things literally it does not fall under a specific secuirty requirement. After all, these secuirty requirements are supposed to help in finding flaws, not in hiding them because a flaw might not fall under some specific security requirement. ).