SoftwareSecurity2014/Group 1/Reflection

Reflections on ASVS

We consider the ASVS a decent standard for performing Web application security verifications, although weaknesses exist. For example, there were individual specifications that were not entirely clear and there was no expanded discussion on them that could clarify a specification’s intent. In addition, ASVS has separate categories that have high overlap with one another (e.g. access control, authentication and session management). (Erik P:I agree these 3 categories are related, but I think they are still largely orthogonal aspects. Access control is the trickiest to pin down in these code reviews, as there is typically no specificaiton of the intended access control requirements and these have to be guessed from the intended functionality, so I might want to steer clear of that one in future projects.)At an abstract level this is understandable because each section taken by itself needs to cover a certain amount of ground. In practice, though, it created some debates as to which category to classify some methods under. In the real world, such overlaps may lead to redundancy of effort between individual task groups, making it more d ifficult to focus resources on one area of code review (since all the requirements one wants are not necessarily in one category) and may reduce a group’s ability to fully cover a given subject area. We do believe, however, that splitting the work into subgroups is sensible, despite the above caveats, and consider sharing between groups to be an important element to the success of the review.

A potential improvement to ASVS is to explicitly acknowledge the existence of such overlaps. If requirement Vx.y partially overlaps with Vp.q, perhaps the standard might collapse them both into a single requirement whenever Vp and Vx are part of the same security audit. This could, on the other hand, make for overly complicated ASVS requirements.

Security requirement guidelines provide some benefit to developers and institutions with little experience with security. There is a risk, however, that such people may assume security is merely a matter of checking off boxes instead of it being a process. A weakness of ASVS is that it is a general guideline that cannot emphasize the most important categories for a specific industry. Banks, for example, would likely place more emphasis on data integrity than access control. Given the limited resources available for security in most firms, such prioritization matters. Furthermore, certain checks in ASVS rely on specific configurations which fall outside the scope of an application. For example, communication security checks (V10) on webapps that are executed by a PHP interpreter and inside an HTTP-server are affected by these extra software packages and their configurations. Requirement V11.2, which checks the that only defined HTTP request methods are accepted, presents another example. It is not clear whether verification should be implemented at the PHP or the server configuration level. We suggest including a sample .htaccess file to enforce the appropriate restrictions, a practice found in other software packages (e.g. Joomla), but question whether this is valid under ASVS’s scope.

The value of a ASVS Level 1 certification (A or B) seems questionable. The potential of such a signal to mislead people with no background in security (managers, for instance) into a false sense of security may be greater than the information it provides to security professionals. The certification may be required for compliance purposes or even insurance purposes but its benefit to an application’s security is small.

We believe a requirement category on code quality would be a valuable addition to ASVS. (Erik P: yes, that's a nice idea)The section might verify that security-sensitive code is concentrated into a few central functions and classes to facilitate development and auditing secure code. Additionally, it should mandate that platform-specific programming best practices are being followed, for example using prepared statements with SQL. The precise requirements for such a section may be difficult to formulate, however, because there are many edge cases to consider and ASVS requirements should not distract from the overarching aim of producing secure code at every line.

As to the overall structure of the standard, we found it difficult to articulate major improvements to ASVS, despite its flaws. This is due in part to the general purpose for which the standard is created. Restructuring the requirements into different categories, for example, may be logical for some applications but not others. Indeed, the modularity of the current categories, although it leads to overlaps in coverage, also allows practitioners to create customized category sets for task delegation according to application and organizational needs. Other major changes may have similar side effects that reduce the standard's usefulness across all applications. In short, it may be easy to poke holes in a standard but much more difficult to devise a robust and flexible alternative.

Reflections on FluxBB

FluxBB is of a manageable size for an assignment of this length and allowed a (nearly) complete review of the critical code sections. Only requirement V11.1, which verifies that no unvalidated data is passed to redirect functions, turned out to be rather elaborate to verify. There were 70 instances of redirects, some using static data, that were easy to verify for format correctness. The relevant variables for the remaining redirects, however, had to be checked for sanitization at their initial assignment statements. We adopted a sampling approach for these cases because many redirects use multiple variables, thus requiring copious reverse data path lookups. In all cases, the inputs were properly sanitized. Verification of this requirement might be easier if the variables used by redirects were sanitized (probably redundantly) just before being used. We are not sure if this is practical, and how much it would cost, but it could have positive side effects - for example, preventing malicious data stored in the database from being used for an attack.

A larger project would have forced a critical assessment of security priorities based on the intended use of the application and the resources available for verification. The tradeoffs and uncertainties that result when focusing on some sections of code and not others may be closer to what security professionals experience.

Reflections on Project

We believe this project was instructive in several regards, but we would suggest some modifications for future classes. Reviewing a project’s code as an outsider demonstrated the importance of good documentation, either in the code or in a wiki, and the benefits of annotations (if available in the language). FluxBB maintains a wiki that is sometimes helpful but also contains many empty function descriptions. Such lapses highlight the need to document code from the beginning.

The importance of moving sensitive information between systems (e.g. to or from a database) in as few code paths as possible was reaffirmed in our verification. In FluxBB, for example, the check_cookie function is the sole point where the session cookie’s HMAC is computed, verified and written so assessing its functionality is concise. It also reduces the attack surface and allows code authors to focus their attention and resources on smaller sections of critical code. In other words, well thought-out modularity and strong adherence to coding style guides improves security.

The value (or lack thereof) of static code analysis tools in aiding a security verification was also instructive. The sheer amount of warnings thrown by Fortify and other applications demonstrated the problems of too many false positives. The developers and vendors of these products would provide a service to security researchers by adding modules that align with security verification standards such as ASVS.

On the critical side, we didn’t understand the value of conducting an assessment using a version of ASVS that is obsolete(Erik P: Note that the ASVS is not all that obsolete - the only newer version only had a beta release this spring. It is not like security has changed - or improved :-) - a lot in the part 5 years... Some folks at OWASP regret the decision to add a year eg to the OWASP Top Ten, because now they are forced to come with an update every few years because people might think the last one is outdated. I'd say that the 2009 ASVS is still on par with other standards being pushed out today by other consortia)) , and only selected portions at that. Using the latest version would help us be up-to-date with industry standards, although we understand that the choice to use the 2009 version was motivated in part to separate static code analysis from dynamic vulnerability scanning. But allowing penetration testing of the application (Level 1A in the ‘09 ASVS version) might demonstrate the limitations of static code analysis since penetration testing tools are sophisticated at finding hard to discover vulnerabilities.

In addition, the particular division of ASVS categories for this assignment seemed curious as alternative groupings suggested themselves. Session management, for instance, seems better grouped with access control than with HTTP security. Allowing project groups to select their own set of categories for review would cause students to reflect more carefully how the different categories are related and to tailor their choice(s) to their chosen project. {{hilite|(Erik P: I'll have to see how things are organised in the new ASVS but thinking btter about the grouping is a good idea. Also, some requirements are less well-suited than others - V10, as you observe rightly, is more of a configuration than code issue. )}

We also believe doing a complete assessment of one web application by each group would provide an beneficial overview of how individual components combine to allow strong statements to be made on the app’s security. In addition, it would reduce redundancy since overlapping requirements would be verified only once.

Concluding remarks

We understand no project is perfect. Our critique does not indicate that we thought the project was poorly designed or that we did not enjoy it. In fact, the opposite is true. Whether the requirement sets were chosen deliberately to make us think about the ASVS structure or whether they were designed as conceptually related areas, we experienced them as thought-provoking and as a launchpad for discussion within the group. We learned a lot from looking at other peoples’ code from a security perspective, and we believe we have gained from having to critically evaluate security goals against a codebase that’s not explicitly setup to accommodate such a review. Although we did not use the latest standard, working with ASVS has given us a taste of real-world security auditing work.(Erik P: Btw, I expect - or rather, fear - that using the latest standard won't improve the taste much...) Overall we are pleased with the scope and organization of the project and the knowledge we gained from the experience.