SoftwareSecurity2012/Group 9/Verdict

== Table ==

This is a summary table of our verdict regarding the requirements. The next sections contain explanations about what has lead us to decide on these verdicts.

Requirement	Verdict
V8.1	Not sure
V8.2	Not sure
V8.3	Satisfied
V8.4	Not sure
V8.5	Failed
V8.6	Failed
V8.7	Satisfied
V8.8	Satisfied
V8.9	Satisfied
V8.10	Satisfied
V8.11	Failed
V8.12	Does not apply

(About the presentation here: Given that several of the requirements are about logging, and - as you point out below - there isn't any logging to speak of, I would say this up front in the overview here, so that before people read the details below it is already clear that several requirements are broken due to the absence of logging)

Requirement 8.1

Description: Verify that that the application does not output error messages or stack traces containing sensitive data that could assist an attacker, including session id and personal information.

The tool did not get any feedback or warning about this requirement. With the analysis we found some SQL-injection issues that might be used to leak information, since there are some queries that can be used to show the table's contents in the screen. However SQL-injections are treated as other requirements. (But of course places where there may be a SQL injection are also places where there might be an error message...)

To try to have more information about this requirement, we did some research on the internet and we manually analyzed the code. By doing that, we found out some PHP error handling patterns that are used to avoid the leaking of information. (Do you mean error handling patterns that are used in fluxbb, or patterns that are used in general (and SHOULD have been used in fluxbb)? Further down it becomes clear you mean the latter)

Verdict: Not sure.

Basic Error Handling 1: Using the die() function

By using the die() function in php, we avoid that output error messages and stack traces containing sensitive data shows up in the screen. For instance we could have this piece of code.

 <?php
 $file=fopen("welcome.txt","r");
 ?>

If the file does not exist you might get an error like this:

 Warning: fopen(welcome.txt) [function.fopen]: failed to open stream: No such file or directory in C:\webfolder\test.php on line 2

To avoid that the user gets an error message like the one above, we test if the file exist before we try to access it:

 <?php
 if(!file_exists("welcome.txt"))
   {
   die("File not found");
   }
 else
   {
   $file=fopen("welcome.txt","r");
   }
 ?>

Now if the file does not exist you get an error like this:

 File not found

The code above is more efficient than the earlier code, because it uses a simple error handling mechanism to stop the script after the error. Looking inside the FluxBB code we did not find any die() function.

Basic Error Handling 2: Set Error Handler function

First of all, the programmer must create a special function (error handler) that is going to be called whenever a error occurs. The error handler function has the following syntax:

 error_function(error_level,error_message,error_file,error_line,error_context)

The default error handler for PHP is the built in error handler. It is possible to change the error handler to apply for only some errors, that way the script can handle different errors in different ways. To add your error function you must call the "set_error_hander()" :

 set_error_handler(My_error_function);

Since we want our custom function to handle all errors, the set_error_handler() only needed one parameter, a second parameter could be added to specify an error level.

Looking inside the FluxBB code we did not find any set_error_handler() function.

Basic Error Handling 3: Trigger an error

In a script where users can input data it is useful to trigger errors when an illegal input occurs. In php, this is done by the trigger_error() function, which creates a user-defined error message and has the following syntax:

 trigger_error("message", 2ndParameter);

An error can be triggered anywhere you wish in a script, and by adding the second parameter, you can specify what error level is triggered.

Possible error types:

 E_USER_ERROR - Fatal user-generated run-time error. Errors that can not be recovered from. Execution of the script is halted
 E_USER_WARNING - Non-fatal user-generated run-time warning. Execution of the script is not halted
 E_USER_NOTICE - Default. User-generated run-time notice. The script found something that might be an error, but could also happen 
                 when running a script normally

We found several files in FluxBB with the trigger_error() function. Meaning that in some point of the code developers were aware about the errors that could show up, but it is not sure if they are all implemented correctly according to the security requirements.

Basic Error Handling 4: Exception handling

Exception handling is used to change the normal flow of the code execution if a specified error (exceptional) condition occurs. This condition is called an exception. When an exception is thrown, the code following it will not be executed, and PHP will try to find the matching "catch" block. If an exception is not caught, a fatal error will be issued with an "Uncaught Exception" message.

To avoid the errors like in the first example, we need to create the proper code to handle an exception. Proper exception code should include:

 Try - A function using an exception should be in a "try" block. If the exception does not trigger, the code will continue as 
       normal. However if the exception triggers, an exception is "thrown".
 Throw - This is how you trigger an exception. Each "throw" must have at least one "catch".
 Catch - A "catch" block retrieves an exception and creates an object containing the exception information.

Looking inside the FluxBB code we did not find any try-catch statements.

Verdict: Not sure.

(Did you try to generate some errors to see what the warning message would be?)

Requirement 8.2

Description: Verify that all server side errors are handled on the server.

The tool have not got any feedback or warning about this requirement. By searching on the internet, we have got almost nothing about this topic. Having only this information and considering that we are using the HTTP protocol, we supose that the error handling in the server-side has the default configuration and all errors, like "404 (Not Found)" or "500 (Internal Server Error)", will be treated as usual on the server. One thing we found was the server-side Error Handling via htaccess in php, by checking the htaccess file in the FluxBB source code we did not find anything special.

Verdict: Not sure.

Requirement 8.3

Description: Verify that all logging controls are implemented on the server.

The tool did not got give any feedback or warning about this requirement. By checking the php.ini file we found out that no logging file will be created by default, to do this we have to fill the error_log = (filename) field. Still, this only writes out the basis error handling output to a file. (What do you mean by "basic error handling output here"? Things like 404 and 500 warnings?) In other words, we can say that no logging controls are implemented at all. Since the requirement states that all logging controls are implemented server side, and this is literally true, we consider the requirement to be satisfied.

Verdict: Satisfied.

Requirement 8.4

Description: Verify that error handling logic in security controls denies access by default.

Error handling is very important because it considers testing abnormal paths which tend not to be executed very often but as we all know to well bugs usually lurk in corners.As an overall impression so far we have seen that error handling is treated pretty good in FluxBB. Default deny is a synonym to white listing and it means that only the persons with an account for the application are being provided access to the application and all the rest are restricted. As an extension of this only people which have the right to access certain information will be authorized to do it and all the others which are unknown to the application are forbidden.We found out that the application seems to be default deny but our opinion is limited to our experience with PHP and to our exploratory "penetration" testing that we conducted on the target application. We looked into the login.php file to see what happens when the password matches or not and we also tried to log in as a normal user and as an administrator to see if a person with lower privileges can access information from a person with higher privileges but this doesn't seem to be the case. We keep on searching for an anomaly in this in order to further clarify whether it indeed is a default deny application or just find a counterexample as soon as possible. For this requirement I would suggest a model - based "penetration" testing approach where the model is a security automaton describing which actions are forbidden for users and which actions are forbidden for administrators (for example it can be that an administrator has a view of all the user contents but for legal matters is not allowed to modify them). I would also suggest that any attempt of elevation of elevation of privileges should be logged also because it is an abnormal situation and logging might be used as direct means of identification or at least as forensic evidence for behavioral profiling of the attacker (in case of hiding behind a proxy). But attempting to manually check all possible combinations of input actions or a full code review (too much dependency - not suited for assigning individual tasks to different team members, too little experience in PHP, too little time for studying a vague documentation - only details about installing FluxBB,tools aren't helping here) is above and beyond the scope of our task.Because these obstacles and because it's very difficult to prove that this property holds completely and beyond doubt my verdict would be : "not sure".

(I understand the problem. The best one can typically do in a manual verifcation is checked in certain places (as you did for login.php) and report if you did/didn't find problems there, but ruling it out in the entire code base is undoable. Note that it could then be regarded as a pass, albeit a qualified one, where you essentially assume the code you inspected (eg. in your case, login.php, is representative of all code.

Verdict: Not Sure.

Requirement 8.5

Description: Verify security logging controls provide the ability to log both success and failure events that are identified as security-relevant.

In case of a successful attack criminals (hackers in our case) usually try to cover there tracks as quickly and effectively as possible so this is one of the main reasons why we need logging: for matters of accountability (non-repudiation). Accountability is the principle that the cause of any event (i.e. a perpetrator) should be both identified and responsible. Moreover it provides feedback to the security policy such that associated vulnerabilities can be found and fixed. Log management is nevertheless complicated due to the size (you can compress them) and complexity of the content. Moreover integrity of logs are a concern (rootkits want modify logs to cover there tracks) but also confidentiality (data protection,anonymity of data).Perhaps because of these obstacles we don't see any logging for the FluxBB the only logging that might be done implicitly by PHP in the command "error_reporting" (see config.ini file) for errors appearing in the browser when running the application.We didn't find traces of error logging whatsoever neither in the code using windows grep searching for keywords corresponding to the functions for error logging (error_log) but neither as an output file generated by the application that would record attempts of successful or unsuccessful log-in sessions (also the size of the disk didn't seem to increase afterwards - this could be hindered by interference of the other applications that modify the size of the disk).

Verdict: Failed.

Requirement 8.6

Description: Verify that each log event includes: a time stamp from a reliable source, severity level of the event, an indication that this is a security relevant event (if mixed with other logs), the identity of the user that caused the event (if there is a user associated with the event), the source IP address of the request associated with the event, whether the event succeeded or failed, and a description of the event.

As FluxBB does not provide an error logging feature, the requirement is not satisfied. Should FluxBB introduce proper logging, we suggest the following scheme for logging an event:

timestamp security_level relevant_flag user source_ip sucfail description

With:

• timestamp: a standard unix time stamp indicating date and time of the event;
• security_level: severity level of the event;
• relevant_flag: an indication that this is a security relevant event;
• user: the identity of the user that caused the event. In case no user triggered the event, use a default value (e.g. no_user);
• source_ip: the source IP address of the request associated with the event;
• sucfail: a flag (either failed or succeeded) indicating whether the event succeeded or failed;
• description: a short description of the event.

Example:

1340349034 SEVERE sec_relevant user_001 92.123.456.78 FAILED access_admin_panel

Verdict: Failed.

Requirement 8.7

Description: Verify that all events that include untrusted data will not execute as code in the intended log viewing software.

Since fluxbb does not have an error logging feature, there is no log to be viewed, which means that no untrusted data will ever execute as code in the intended log viewing software. Therefore, the requirement is satisfied.

Verdict: Satisfied.

Requirement 8.8

Description: Verify that security logs are protected from unauthorized access and modification.

FluxBB does not create log files in contrast to other comparable software (Unclassified Newsboard [1]).

PHP can create log files, but it is out of the range of FluxBB to check/modify their access/modification authorization. In a default installation of PHP they are protected by the Linux file system ACLs.

Verdict: Satisfied.

Requirement 8.9

Description: Verify that there is a single logging implementation that is used by the application.

First, it should be noted, that FluxBB does not output log files on its own. So with regard to that, the requirement, that not more than one function is used, were satisfied.

On the other hand we have found the following functions being used for Error Handling and Logging in PHP in general. Looking through the code we have found the following in files:

die()

debug_backtrace()

debug_print_backtrace()

error_get_last()

error_log()

try(), throw() and catch()

restore_error_handler()

restore_exception_handler()

set_error_handler()

set_exception_handler()

all of them nothing found

error_reporting()

philipp@linux-s7jg:/srv/www/htdocs/fluxbb-1.4.8> grep -lir error_reporting *

db_update.php

include/common.php install.php

trigger_error()

philipp@linux-s7jg:/srv/www/htdocs/fluxbb-1.4.8> grep -lir trigger_error * include/utf8/utils/unicode.php include/utf8/utils/bad.php include/utf8/mbstring/core.php include/utf8/ord.php include/utf8/native/core.php include/utf8/utf8.php include/utf8/str_pad.php

user_error()

philipp@linux-s7jg:/srv/www/htdocs/fluxbb-1.4.8> grep -lir user_error * include/utf8/native/core.php include/utf8/utf8.php include/utf8/str_pad.php

So within FluxBB software user_error(), trigger_error() and error_reporting() (more than one) are used. So the requirement seems to has failed.

A more detailed analysis of the code shows, that user_error() function is never called, but that is was part of the following:

trigger_error('String functions are overloaded by mbstring', E_USER_ERROR);

Searching what error_reporting() function does, it shows, that this simply turns on PHP error reporting while execution, but does not log anything triggered by the software:

// Turn on full PHP error reporting

error_reporting(E_ALL);

So with regard to that, only one single function is used: trigger_error(). Strangely this is only an included piece of software not by the code writers themselves.

Verdict: Satisfied.

Requirement 8.10

Description: Verify that the application does not log application-specific sensitive data that could assist an attacker, including user’s session ids and personal or sensitive information.

FluxBB does not log any events at all. Thus, no application-specific sensitive data is logged and the requirement is satisfied.

Verdict: Satisfied.

Requirement 8.11

Description: Verify that a log analysis tool is available which allows the analyst to search for log events based on combinations of search criteria across all fields in the log record format supported by this system.

No "log analysis tool is available which allows the analyst to search for log events based on combinations of search criteria across all fields in the log record format supported" by FluxBB, simply because there is no log record at all. Hence, FluxBB fails to meet this requirement.

Should there be sufficient logging in FluxBB, meeting the criteria as mentioned in 8.6, we suggest the use of log analysis tools like OSSEC[2], LogHound[3] and SLCT[4]. They provide pattern search algorithms for large-sized log files, which could be very useful for analyzing frequent attackers or exploits for example.

Verdict: Failed.

Requirement 8.12

Description: Verify that all code implementing or using error handling and logging controls is not affected by any malicious code.

This requirement does not apply for level 1 or level 2, meaning it was beyond the scope of the project.

Verdict: Does not apply.

Extra information

The default behavior of the program can be also described in configuration file php.ini. In it, we have some error handling and logging configuration options that can be changed according to the programmer's desire. The following table shows more information about what can we do with the php.ini file for our requirements and what are the default values.

Function	Description	Default value
error_reporting	Set the reporting level when a error is found	E_ALL
display_errors	Print out errors (as a part of the output)	Off
display_startup_errors	Even when display_errors is on, errors that occur during PHP's startup	Off
log_errors	Log errors into a log file (server-specific log, stderr, or error_log )	On
log_errors_max_len	Set maximum length of log_errors	1024
ignore_repeated_errors	Do not log repeated messages	Off
ignore_repeated_source	Ignore source of message when ignoring repeated messages	Off
report_memleaks	If this parameter is set to Off, then memory leaks will not be shown	On
track_errors	Store the last error/warning message in $php_errormsg (boolean)	Off
error_log	Name of the file where script errors should be logged	disabled