SoftwareSecurity2013/Group 2/Reflection

One page reflecting on the whole OWASP ASVS experience, eg discussing issues such as

What difficulties did you encounter, either with the code or the ASVS? Can you think of ways to reduce or avoid these difficulties?

group:

Fortify is a great tool for finding security sensitive code and check if this code is called from a possible insecure location. But for our requirements Fortify itself was not very useful. Access control is hard to find by a code analyzer and also Fortify’s analysis did not know about access requirements or rights and so it could not find these errors. As a result we had to look at the PHP code manually. What made all this difficult is that the codebase is far too large to go through when you know absolutely nothing about the code.

(Maarten) The requirements we chose were a bit hard to evaluate with static code analysers. So we still had to do a lot of stuff by hand.

(Nick) Access control is hard to find by a code analizer. He cannot find whether a user is logged in and cannot figure out which code is available for all users and which only for admins. It is a great tool for finding security sensitive code and maybe check if this code is called from a insecure location. Because we chose access control the tools where not very useful.

(Dorus) The fortify analyse was mostly useless, fortify does not know about access requirements or rights so it can not find these errors. As a result I had to look at the PHP code manual, but my experience with PHP is very limited and the Budypress code is very organized (spaghetti code). Half the security checks depend on additional checks and calls to checking functions all over the code, as well as assumptions on what Wordpress (hopefully) checks on.

(Karin) The chosen requirements were hard to check with a static code analyser tool. This results in do the analyses manually instead of automatically with Fortify. For example you cannot see if a function or an url are required to have authorization. You have to know the specifications to look when authorization is missed to mention or all users can access it. To my idea you cannot avoid this problems, because to my idea you have to check it in running time. Fortify cannot do this.

(Ennier) The problem with the code, is that the codebase is far too large to go through when you know absolutely nothing about the code. Furtermore, it is difficult to actually navigate the source without a decent code editor. This can be mitigated by getting a better code editor or a full fledge IDE that can do code analysis.

Fortify itself was not very usefull when it came to rights management for services or data that can be seen. This is because it does not know of the business requirements. There is no real way to avoid this difficulty unless Fortify builds a way to give the requirements.

Could the ASVS be clearer, more complete, or structured in a better way?

samen The ASVS are clearly specified, complete, structured and point in the right direction for many common problems. However there is a lot of overlap between requirements, sometimes it's tempting to classify a problem in Access Control as a Authentication problem, or vica versa. We think ASVS is quite useful when evaluating the security of your program, but you have to adapt your entire development process to it if you want to get the most out of them.

(Nick)The standard is quite oke but some of the requirements are quite close to each other. Because of this some of the results will be almost the same.

(Maarten) I think the ASVS is pretty clear, complete and structured. It has been around for a while, so it had some updates and refinements in the last year. I think it's quite useful when evaluating the security of your program, but you have to adapt your entire development process to it if you want to get the most out of them.

(Dorus) The ASVS are clearly specified and point in the right direction for many common problems. However there is a lot of overlap between requirements, sometimes it's tempting to classify a problem under Access Control as a Authentication problem, or vica versa. It is probably correct only one of these two classes applies, but it's hard to decide which on. At the same time, an requirement like "Verify that the same access control rules implied by the presentation layer are enforced on the server side." might look very detailed, it also require you to check almost all of the project code. Even if an access is checked at one point, an function deeper in the code might still make incorrect access possible by mistake. These kind of functions are impossible to check on manual, and would probably be better fit to check with an automated tool. Even an automated tool like that would still be very complicated (and thus error prone), and would depend on meta info like annotations to know what roles exists and who can access that. This meta info can, again, be wrong itself. And better specification in my opinion would be "Verify that all access controll rules are enforced by the server side" - its kinda mood if the client side does any verification at all, client side verifications are only for the users convenience and add nothing to security. Even worse, in some situations the user side might be more strict then absolutely required. For example the administrator might not have a button wipe an entire board, but a hidden option in the code that allows only administrators to do this impose no problems at all.

(Karin) I think the standard is very good. There are some general requirements, which contain more detailed point what to check. Especially those detailed points were a good guideline what to check. It was very clear. The only problem I saw that it was sometimes a problem would fit in more than one requirement. To my opinion ASVS is complete, but possibly in the future there have to be added more points (security problems changes).

(Ennier) I thought ASVS was pretty clear on the warnings it gave and I liked how they can show warnings in categories. I have no other complaints about it. It would be nice to have some syntax highlighting when viewing the code and the ability to jump to the definiton of functions. This should be possible seeing that they do code analysis and code flow.

Is splitting up the work as we did,with different groups looking at different security requirement, a sensible way to split the work? Or are there more practical ways?

samen We think it would be better if multiple groups where looking at the same tools using the same requirements. This makes it easier to compare results. Comparing results makes it easier to find bugs group A found and B didn't and the other way around. From this difference I think you can learn more from each other. Because we ran into problems that where hard to classify under specific requirements, it might be an idea to allowing groups to go over multiple requirements and simply document all problems they run into. This might result in finding more problems. At the same time it's also easier this way to simply go for the low hanging fruit.

Ik heb het idee dat Ennier en Karin het betrokken hebben op onze eigen groep. Hier wordt bedoeld of het bevallen is dat iedere groep van de klas dus naar een ander onderwerp kijkt en naar andere requirements.

(Nick) I think it would be better if multiple groups where looking at the same tools using the same requirements. This makes it easier to compare results. Comparing results makes it easier to find bugs group A found and B didn't and the other way around. From this difference I think you can learn more from each other.

(Maarten) Maybe it would have been better to have multiple groups looking at the same verification requirements (and maybe even at the same code!) to see if they have comparable results. This way you can check if they learned the same lessons. Maybe you chose a webapp for which your security requirements are hard to evaluate, so learn other things than other groups.

(Dorus) As i mentioned before, i ran into problems that where hard to classify under specific requirements. Allowing groups to go over multiple requirements and simply document all problems they run into might result in finding more problems. At the same time it's also easier this way to simply go for the low hanging fruit. That might be both good and bad. Right now you are forced to look at problems that might seem less appealing at first. Going trough the requirements one at the time one requirement at the time might seem like an organised way to search for problems, at the same time i got the feeling we could work more focussed if we instead split the codebase into a number of logical parts, and really extensively look at certain parts of the code and try to completely understand this part. One student might look at the page where you write blogposts and view them, another might look at the comment section and all related codefiles. This approach might work better for some of the requirement then others. Right now i had the feeling our codebase was too large to completely understand and check, also related to my limited experience with PHP it was hard to do any real digging into the code.

(Karin) To my opinion the work could not split very well, because some problems will fit more requirements. Besides this, you could do better the same requirements, so you can compare the results (especially when you have to do it by hand).

(Ennier) Inside an orginazation that is a sensible way of splitting the work. In the way this project is done, it would be a good idea to keep some of the parameters of the project the same. However, I thought that the different requirements are pretty interesting to do.

What could or should developers do to facilitate a security assessment? Do you think that experiencing security from the perspective of a reviewer rather than a developer has changed the way you would design or implement a web application yourself?

samen Security assessments should be thoroughly interweaved with a company's development method. Normally as programmer you really focus on implementing the function specifications, but if you never think about security and afterwards do a review, then it is harder to make the application secure in a proper way. It is also handy to do certain (security) checks in a central place. On an webapp you do not want every single page to have direct access to background variables, this should much rather go through and abstracted layer where you can check for problems in an organized pattern.

(Maarten) I think security assessments should be thoroughly interweaved with a company's development method. You should start thinking about security immediately from the beginning of a project. So develop security requirements and model the way they should be implemented. This way, when doing reviews, you can look back at what you had designed and check if this is implemented correctly. If you never think about security and then do a review, you're already to late. You won't know what and what not to fix. So you could say that experiencing security from the perspective of a reviewer rather than a developer has changed the way I would design or implement a web application.

(Dorus) Looking at code from a reviewer perspective first of all made me realize i do not like to spend too much time looking at other peoples code, especially PHP. I much rather understand one piece of code very well then having to look at the overall structure of an entire project. It also confirmed what i already knew, it's important to have a well layout plan for larger codebases. It is especial important to do certain (security) checks in a central place. On an webapp you do not want every single page to have direct access to background variables, this should much rather go trough and abstracted layer where you can check for problems in an organized pattern.

(Karin) I think as a company, you have to have a special focus on security aspects. Normally as programmer you really focus on implementing the function specifications, but you have to change that. This change could be made by a security assessment.

(Ennier) The security aspects of an application should be considered from the very beginning of the project. I think that central modules should be implemented that there is only one particular way of implementing the security requirements. It has therefore in some ways change my views, but in implementing a web application not much has changed. These things are already things you should be doing to begin with.

Any other interesting, frustating, boring, educational, etc. aspects of the whole experience.

(Nick) The hole project was bigger then I suspected at first. The plug-in we examined was bigger and the code analizers did not help a lot for finding possible bugs, which made it necessary to get into the code. Because the code was big and had a lot of cross-references to other pieces and pieces of code of wordpress it took an awful long time the check a whole flow. There is also a positive side on this, because of the research I learned some new functionality in PHP and tricks you can do with PHP.

(Dorus) My biggest frustration was my limited knowledge of the background systems like wordpress and PHP, but also the exact functionality of buddypress. This made it close to impossible to say "this requirement pass", i can easily check if a certain specific check is made, but it's much harder to see if this check is NOT evaded by mistake anywhere else in the code, or even if the check itself is implemented sound. (Erik:I guess that in retrospect that it would be less frustating to let people not evaluate plugins, but self-containted web-apps.)

(Karin) I learned that a tool could not analyse the code on some aspects very well. You have to search for problems also by hand. Besides that the project showed how important it is to know the system and its specification. I think it was interesting to learn a new tool and knowing the possibilities of such a tool.