Grafana + Prometheus = Awesome

The first blog post for setting up Prometheus and adding a custom client library is here.

As a quick reminder, the custom metrics I have is the count of API calls by Method and Endpoint.

Custom Metrics

Grafana

Grafana is an excellent graphing data visualization platform. Go ahead and download it from here.

Start it with this command.

.\grafana-server.exe web

PS C:\Users\Daniel\Downloads\grafana-5.2.1.windows-amd64\grafana-5.2.1\bin> .\grafana-server.exe web
...
...
...
INFO[07-24|19:04:18] Initializing SearchService               logger=server
INFO[07-24|19:04:18] Initializing PluginManager               logger=server
INFO[07-24|19:04:18] Starting plugin search                   logger=plugins
INFO[07-24|19:04:18] Plugin dir created                       logger=plugins dir=C:\\Users\\Daniel\\Downloads\\grafana-5.2.1.windows-amd64\\grafana-5.2.1\\data\\plugins
INFO[07-24|19:04:18] Initializing InternalMetricsService      logger=server
INFO[07-24|19:04:18] Initializing AlertingService             logger=server
INFO[07-24|19:04:18] Initializing HTTPServer                  logger=server
INFO[07-24|19:04:18] Initializing CleanUpService              logger=server
INFO[07-24|19:04:18] Initializing NotificationService         logger=server
INFO[07-24|19:04:18] Initializing ProvisioningService         logger=server
INFO[07-24|19:04:18] Initializing RenderingService            logger=server
INFO[07-24|19:04:18] Initializing TracingService              logger=server
INFO[07-24|19:04:18] Initializing Stream Manager
INFO[07-24|19:04:18] HTTP Server Listen                       logger=http.server address=0.0.0.0:3000 protocol=http subUrl= socket=

The default username and password is "admin" and "admin". The url is http://localhost:3000/login. On first login, you might be prompted to change password.

Grafana Login

On first login, there won't be anything to show.

Grafana Empty

Go ahead and click on that "Add data source". But make sure that Prometheus is running on the local machine.

Fill out Grafana Data source as shown below to add a localhost Prometheus, then "Save &d Test".

Grafana Data Source

Choose "New Dashboard" from the side button.

Grafana New Dashboard

Let's create a new graph panel.

Grafana New Dashboard

Grafana Edit Dashboard

Fill out the textboxes with below. Also, make sure to do whatever needed to generate custom metrics.

  • Formula: rate(PathCounter[5m])
  • Legend Format: {{method}} - {{endpoint}}

Grafana Rate

On the top, there's a "Save button" icon. Click

Save As

Congratulations! The dashboard now has a fancy graph showing data in real-time.

Save As

Summary

Add dashboards, add panels, add visualizations. Customize the view of data however you need to. Make the data actionable and valuable for your boss, for your team, and for you. Data visualization without meaning is just white noise.