Monitoring With Prometheus: How It Works

An IT product without metrics is like a ship without a rudder and sails. Metrics need to be collected, analyzed, and only based on their values should decisions be made. Let’s talk about Prometheus, how it collects metrics and why standard databases are not always suitable for this.

How Did Prometheus Come About?

Running a business without knowing the numbers is impossible. To make informed decisions, you need to know the key business metrics: revenue, profit, number of orders, number of active users, plus as many less significant parameters as possible: the distribution of user activity by region, the most popular products on the site, server load. By owning all this analytical information, you can understand where you are in the sea of ​​business and where you need to row.

In 2012, the online audio distribution platform SoundCloud was looking for a new analytics tool. After weighing all the options and trying everything that was on the market at that moment, the company began to develop its solution.

At that time, analytical DBMS were either too complex or too unreliable and slow. At the same time, the Go language began to gain popularity. The system for collecting metrics on Go could become an order of magnitude cooler than its counterparts, so the developers from SoundCloud decided to write their tool on it.

The decision turned out to be good: now Prometheus is the fastest and most powerful system for collecting analytics.

Why Can’t You Use, Say, MySQL To Collect Metrics?

Can! If you have a small project, a few values ​​in analytics, and the data is rarely updated, as much as possible. Standard DBMS cannot provide the required performance and reliability in other cases. All these cool things like indexes and transactions, which are essential in some other cases, only slow down the database, which requires maximum speed and reliability in recording metrics.

Therefore, for analytics systems, they developed the concept of a time series database – a system for storing simple values ​​\u200b\u200b bound to specific points in time. Any base with this approach will have two dimensions – a moment in time and the values ​​themselves. Such a straightforward approach to the structure of stored information allows you to plan algorithms for searching and processing values and squeeze out the maximum speed and reliability. The space of states in which such a DBMS can be located is minimal and well described.

This approach to the internal architecture of the database allows you to quickly record a large number of different metrics and run tricky statistical queries that show the dynamics of these metrics over time. Since these operations are the only purpose of Prometheus, they are performed as quickly as possible. Conventional DBMS are designed for a wide range of data recording and processing tasks, so they lose speed when working with sets of metrics.

Also Read: Data Storage & How It Can Be Managed