Prometheus rev 01

From Open-E Wiki
Revision as of 10:06, 30 June 2026 by Ai-B (talk | contribs) (Create Prometheus extension article rev 01 (xc-prometheus SU))
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Prometheus is a metrics database packaged as an LXC container Small Update for the storage server. Once installed, it continuously collects system, ZFS storage, and hardware sensor data from the appliance and makes it available for querying and visualisation.

Note: Prometheus is delivered as an optional Small Update (the xc-prometheus module). This article describes revision 01. If your system was updated to a newer revision, refer to the matching Extension:Prometheus_rev_NN article.

How it works

Prometheus scrapes metrics from exporters running on the storage server (provided by the x-metrics-exp companion module) and stores them in its time-series database (TSDB). The Grafana extension (xc-grafana) then reads from Prometheus to render dashboards in the browser.

The full data flow is:

  • The storage server runs a Node Exporter (system metrics: CPU, memory, disk, and network, plus ZFS pool metrics via a textfile collector) and an IPMI Exporter (hardware sensor metrics).
  • Prometheus scrapes the exporters every 15 seconds and writes the samples to its local TSDB.
  • Grafana reads from Prometheus and renders the dashboards in the browser.

All three modules (x-metrics-exp, xc-prometheus, and xc-grafana) must be installed for the dashboards to display data.

Scrape targets

Prometheus is pre-configured with three scrape jobs:

Job Target Description
prometheus localhost:9090 Prometheus self-monitoring (scrape interval: 5 s)
node localhost:9100, localhost:9290 Node Exporter — system metrics (CPU, memory, disk, network) and ZFS pool metrics via textfile collector
ipmi localhost:9290 IPMI Exporter — hardware sensor readings (temperatures, fan speeds, voltages, power)

Reference

Item Value
Web UI / API address 127.0.0.1:9090 (loopback only — not externally accessible)
Protocol HTTP
TSDB data directory (inside container) /var/lib/prometheus
Prometheus configuration (inside container) /etc/prometheus/prometheus.yml
Scrape interval 15 s (5 s for self-monitoring)
Memory limit 1024 MB

Maintenance

Checking service status

From the storage server shell, check whether the Prometheus container is running:

 ssh localhost lxc-ls -f

The prometheus container should show state RUNNING.

Viewing logs

Connect to the container console and inspect the Prometheus service log:

 journalctl -u prometheus --no-pager -n 50

Updates

Do not run system package updates inside the container. Updates are delivered as new container revisions through the standard small-update mechanism.

Known issues and limitations

  • All three companion modules must be installed. Prometheus requires the x-metrics-exp Small Update to have any metrics to scrape. Grafana (xc-grafana) requires Prometheus to be running to display dashboards. If any module is missing, dashboards will show No data for all panels.
  • Web UI and API are loopback-only. The Prometheus web interface and PromQL API are bound to 127.0.0.1:9090 and are not accessible from external hosts. This is intentional — Prometheus is a backend service accessed only by Grafana and for local troubleshooting from the storage server shell.
  • No in-container package updates. Running apt upgrade or similar inside the container is not supported due to disk space constraints. Updates are delivered as new container revisions.

Changelog

rev 01

  • Initial release: Prometheus packaged as a standalone LXC container Small Update (xc-prometheus module), separated from the combined Grafana/Prometheus stack.
  • Prometheus web UI and API bound to 127.0.0.1:9090 (loopback only).
  • Pre-configured scrape targets: Prometheus self-monitoring, Node Exporter (system + ZFS metrics), and IPMI Exporter.

For further customization or troubleshooting, refer to the upstream Prometheus documentation or contact Open-E support.