Logging Strategy For Software Applications

Tarun Pabbi / February 19, 2020

Diagnostic logging within the software applications was a powerful tool in the initial stages of the software development, where we had very few IDE’s available and even lesser with debugging support. The best way to know your code execution was by adding log statements in the flow and then analysing the logs. Even recently we follow the same approach for client-side debugging in the browser by looking at the log statements in the console. But due to new powerful debugging tools and now even remote debugging capabilities in new IDE’s like Visual Studio we put less effort on log statements. Mostly they are added only after an issue occurs in the deployed version, due to which the quality and focus of the log statement tend to be towards the problem in hand instead of the holistic approach towards code execution.

Another issue we encountered with logging is the performance impact. Usually, when we require a high-performant code adding too many log statements tend to slow down the application performance due to a lot of I/O operations depending on which log store you are using. Therefore, it is important that you define your log strategy right at the beginning of your project planning and not to delay it towards the end of your development cycle.

Some of the key concepts for a logging system are explained below:

Log Levels

Each log level has its own significance during the troubleshooting process. Using proper log level ensures you are able to find appropriate message during troubleshooting.

 Error

An error or critical is used to log all unhandled exceptions. This is typically logged inside a catch block at the boundary of the application/service. Ideally, you write a global Exception Handler to handle these types of errors.

Warning

A warning is often used for handled exceptions i.e. scenarios where we can recover automatically from within the application. For example, if we are fetching some value and it fails for that process, but we have a default value defined than we can use warning log level to indicate a possible warning scenario. Possible scenarios include Fallback code where we are using default values.

Informational

Generally useful information to log (service start/stop, configuration assumptions, etc.). Information that is diagnostically helpful during troubleshooting.

Debug

This generally includes tracing information. It is usually implemented using assembly level static analysis tools like  PostSharp. Or you can write custom code to write debug level log statements. This level is not used for a production environment but only in lower environments.

Log critical section of code

Most important code paths in an application which determine successful/failure execution of each module should be logged using info log level statements. This makes sure that you can identify issues easily and rapidly during troubleshooting.

But keep in mind that this can easily go out of hand when you start adding logs in each and every method and flows, for example, CRUD operations or entry/exit of methods. Generally, most of the logging in web/app layer will be handled at common methods like creating containers, creating DB connections, calling WCF services etc.  However, in case of background services, there might be special cases like initialisation of services, external components connections etc. which should be logged separately.

Context Information

All the log statements should have proper context information logged with it. This could be userId or Application-Id or anything which clearly defines the context in which this log statement was executed. But this could lead to potential PII data leakage which should clearly be handled to make sure there is no unwanted data in logs.

Default Log Level

Usually, we keep default log level based upon the type of environment we are tar-getting. For example:

• Production – Error Level
• UAT/Staging – Info Level
• Performance – Error Level
• Test/Dev – Debug or verbose

Logs purging

You should also have a log purging strategy in place in order to manage the size of logs generated. For example, you can run a periodic job to flush out older logs or if you using a logging framework like Log4Net, they provide configuration based log purging.

Logging Framework

There are tons of framework and libraries available to satisfy logging needs of modern applications. You can leverage any of these for your requirement based on programming language and hosting platform, but make sure to create a wrapper class using these so that you can easily replace them with some other framework should the need arises.

Conclusion

It is very important to have your logging strategy defined during planning phase so that you save effort and time during your application development lifecycle.