Monitoring your server can sometimes be difficult, but it's essential for you to monitor your server to know which apps are actually using the most of your ressources, is your machine running out of memory ? is it time to upgrade or attach a new volume ?

This article will help you scaffold your Monitoring using Prometheus & Grafana to get a working monitoring solution and in less than 10 minutes, bonus you don't need to use Grafana in your machine since Grafana Cloud offers a free tier.

You'll be able to monitor your Linux Machine in the end of this article and be able to understand how does it work

Chapter 1 : Initializing prometheus & the node-exporter using Docker

The Docker-compose file

version: "3.8"

services:
  node_exporter:
    image: quay.io/prometheus/node-exporter:latest
    container_name: node_exporter
    command:
      - "--path.rootfs=/host"
    network_mode: host # port 9100 is exposed by default
    pid: host
    restart: unless-stopped
    volumes:
      - "/:/host:ro,rslave"
  prometheus:
    image: quay.io/prometheus/prometheus:latest
    container_name: prometheus
    network_mode: host
    volumes:
      - "./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml"

Node exporter ?

But what is an exporter ?

An exporter is software that you installs in your machine, it collect data metrics from something and export it - or make it available for another service to use it for monitoring or analytics purposes in this case Grafana dashboards

What is Prometheus ?

Prometheus is a time series database used for collecting metrics you would export using an exporter, you can think of it as big monitoring software block that collect multiple sources of metrics from anywhere and then store it time-based and provides you a way to query on these metrics, the goal at the end is to link your big prometheus " monitoring tower " to your Grafana instance

What is Node exporter ?

It is one of the "exporters" that Prometheus provide you with, The Node one is used to collect system data, like :

• bootime
• Cpu Frequency
• Ram Usage
• Network requests
• Storage Usage
• More

Chapter 2 : Understand what is going on

Prometheus.yml config file

global:
  scrape_interval: 15s # By default, scrape targets every 15 seconds.

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
    monitor: "codelab-monitor"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    static_configs:
      - targets: ["127.0.0.1:9090"]

  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "node-exporter"

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    static_configs:
      - targets: ["127.0.0.1:9100"]

How will it work ?

• Node exporter will collect machine metrics and will export it via a port :9100
• Prometheus will be initialized and active in port :9090 we'll add the port :9100 to the list of target to check
• Prometheus will be able to get data from the Node exporter since we explicitly listed the URI Port to check in the prometheus.yml
• When accessing the Prometheus web page we'll be able to see the metrics in a time table

Back to Docker

You probably noticed that I used the network mode host, that's because Node exporter need to access the host system, you can find more informations about that here

Chapter 3 : Let's do it

Prometheus

• In a folder, copy the docker-compose.yml contents to a docker-compose.yml file
• Create a prometheus folder in your root folder
• In your prometheus folder create a prometheus.yml config file and copy the prometheus config below
• Initialize the docker-compose

docker-compose up -d

• Your Docker config should be working and everything up
• Check that your targets are up in the prometheus dashboard

• You can now explore all the metrics that the Node exporter provides you with

• Here for example we can see memory usage by time

Grafana

Grafana offers a free tier of their Hosted instances, just grab one at Grafana.com,

• Grafana probably already installed a Dashboard in your account you can delete it or just keep it
• Now you have to link your Prometheus that you installed in your machine to Grafana
• Go to Configuration > Data Sources
• Choose Prometheus
• In URL put the URL of your Prometheus service it should have the port :9090 and start with http
• Choose Access : Server
• And your good to go !

When using grafana you can either choose to make your own panels/dashboards OR use Dashboard templates, we'll use a pretty useful Grafana dashboard that I found complete : this one

• Go to Grafana
• Click on the "+" Icon
• Select Import
• Put the Grafana dashboard URL or ID (1860)
• Click on load

Congratulations 🥳 , you know have a complete dashboard that monitor your linux machine and it should looks like this :