To support the application, there are auxiliary components, such as API gateways, load-balancers, and messaging brokers. The UI is likely backed by multiple RESTful services, possibly built in Java Spring Boot or Python Flask, and a database or databases, such as MongoDB or MySQL. The frontend UI might be built with Angular, React, or Node. In the context of this post, log aggregation and visualization is defined as the collection, centralized storage, and the ability to simultaneously display application logs from multiple, dissimilar sources. The solution is log aggregation and visualization. Given the average complexity of today’s modern, distributed, containerized application platforms, accessing individual logs is simply unrealistic and ineffective. Hopefully, you are still not in this situation.

What was often a simple problem to diagnose and fix, became an unnecessarily time-consuming ordeal. Since most enterprise applications are composed of individual components running on multiple application and web servers, it was necessary to request multiple logs. Requesting and obtaining access to logs typically took hours or days, or simply never got approved.

Many organizations I’ve worked with have created elaborate systems for requesting, granting, and revoking access to application logs. As a Developer and DevOps Engineer, it wasn’t that long ago, I spent a lot of time requesting logs from Operations teams for applications running in Production.

Docker syslog tag file