Prometheus rev 01

From Open-E Wiki
Jump to navigation Jump to search
Ai-B (talk | contribs)
Create Prometheus extension article rev 01 (xc-prometheus SU)
 
Ai-B (talk | contribs)
Correction: remove shell commands, ports, and internal file paths not reachable by GUI-only end users; reframe as extension dependency on Grafana/Metrics exporters; add user-facing storage/retention facts
Line 1: Line 1:
__NOTOC__
__NOTOC__
Prometheus is a metrics database packaged as an LXC container Small Update for the storage server. Once installed, it continuously collects system, ZFS storage, and hardware sensor data from the appliance and makes it available for querying and visualisation.
Prometheus is a metrics database that works in the background as part of the storage server's monitoring feature. Once installed, it continuously collects system, storage, and hardware sensor data and makes that data available to the dashboards provided by the Grafana extension.


'''Note:''' Prometheus is delivered as an optional Small Update (the '''xc-prometheus''' module). This article describes revision '''01'''. If your system was updated to a newer revision, refer to the matching ''Extension:Prometheus_rev_NN'' article.
'''Note:''' Prometheus is delivered as an optional Small Update (the '''xc-prometheus''' module). This article describes revision '''01'''. If your system was updated to a newer revision, refer to the matching ''Extension:Prometheus_rev_NN'' article.
Line 6: Line 6:
== How it works ==
== How it works ==


Prometheus scrapes metrics from exporters running on the storage server (provided by the '''x-metrics-exp''' companion module) and stores them in its time-series database (TSDB). The Grafana extension ('''xc-grafana''') then reads from Prometheus to render dashboards in the browser.
Prometheus is a companion extension for the monitoring dashboards feature — it has no page of its own in the GUI. It works together with two other extensions:


The full data flow is:
*The '''Metrics exporters''' extension, which gathers system, storage, and hardware sensor data on the storage server.
*The '''Grafana''' extension, which reads the collected data from Prometheus and displays it as dashboards in your browser.


*The storage server runs a Node Exporter (system metrics: CPU, memory, disk, and network, plus ZFS pool metrics via a textfile collector) and an IPMI Exporter (hardware sensor metrics).
All three extensions — Metrics exporters, Prometheus, and Grafana — must be installed for the dashboards to display data.
*Prometheus scrapes the exporters every 15 seconds and writes the samples to its local TSDB.
*Grafana reads from Prometheus and renders the dashboards in the browser.


All three modules ('''x-metrics-exp''', '''xc-prometheus''', and '''xc-grafana''') must be installed for the dashboards to display data.
Once installed, Prometheus runs automatically in the background. There is nothing to configure, and no separate login or user interface — all interaction with the collected metrics happens through the Grafana dashboards.


== Scrape targets ==
To view the metrics that Prometheus collects, open the dashboards provided by the [[Extension:Grafana_rev_03|Grafana extension]].


Prometheus is pre-configured with three scrape jobs:
== Known issues and limitations ==
 
{| class="wikitable"
! Job !! Target !! Description
|-
| <code>prometheus</code> || <code>localhost:9090</code> || Prometheus self-monitoring (scrape interval: 5 s)
|-
| <code>node</code> || <code>localhost:9100</code>, <code>localhost:9290</code> || Node Exporter — system metrics (CPU, memory, disk, network) and ZFS pool metrics via textfile collector
|-
| <code>ipmi</code> || <code>localhost:9290</code> || IPMI Exporter — hardware sensor readings (temperatures, fan speeds, voltages, power)
|}
 
== Reference ==
 
{| class="wikitable"
! Item !! Value
|-
| Web UI / API address || <code>127.0.0.1:9090</code> (loopback only — not externally accessible)
|-
| Protocol || HTTP
|-
| TSDB data directory (inside container) || <code>/var/lib/prometheus</code>
|-
| Prometheus configuration (inside container) || <code>/etc/prometheus/prometheus.yml</code>
|-
| Scrape interval || 15 s (5 s for self-monitoring)
|-
| Memory limit || 1024 MB
|}
 
== Maintenance ==
 
=== Checking service status ===
 
From the storage server shell, check whether the Prometheus container is running:
 
  ssh localhost lxc-ls -f
 
The '''prometheus''' container should show state '''RUNNING'''.
 
=== Viewing logs ===


Connect to the container console and inspect the Prometheus service log:
*'''All companion extensions must be installed.''' Prometheus has no data to collect without the Metrics exporters extension, and no dashboards can display its data without the Grafana extension. If any of the three extensions is missing, the dashboards will show ''No data'' for all panels.


  journalctl -u prometheus --no-pager -n 50
== Storage and data retention ==


=== Updates ===
The metrics database automatically keeps roughly the last '''15 days''' of measurements; older data is discarded automatically, so there is nothing to clean up.


Do not run system package updates inside the container. Updates are delivered as new container revisions through the standard small-update mechanism.
It uses a dedicated storage area of about '''1&nbsp;GB'''. Under normal operation, 15 days of metrics use only a fraction of that space, so the database does not fill up in typical use.
 
== Known issues and limitations ==


*'''All three companion modules must be installed.''' Prometheus requires the '''x-metrics-exp''' Small Update to have any metrics to scrape. Grafana ('''xc-grafana''') requires Prometheus to be running to display dashboards. If any module is missing, dashboards will show ''No data'' for all panels.
If the storage area ever fills — for example after an unusually high volume of metrics — the database temporarily pauses recording new measurements until space is automatically reclaimed as the oldest data ages out. No user action is required; it recovers on its own. The only visible effect is a gap in the dashboards for the affected period, and existing data and other services are unaffected.
*'''Web UI and API are loopback-only.''' The Prometheus web interface and PromQL API are bound to <code>127.0.0.1:9090</code> and are not accessible from external hosts. This is intentional — Prometheus is a backend service accessed only by Grafana and for local troubleshooting from the storage server shell.
*'''No in-container package updates.''' Running <code>apt upgrade</code> or similar inside the container is not supported due to disk space constraints. Updates are delivered as new container revisions.


== Changelog ==
== Changelog ==
Line 78: Line 33:
=== rev 01 ===
=== rev 01 ===


*Initial release: Prometheus packaged as a standalone LXC container Small Update ('''xc-prometheus''' module), separated from the combined Grafana/Prometheus stack.
*Initial release of the Prometheus metrics-database extension.
*Prometheus web UI and API bound to <code>127.0.0.1:9090</code> (loopback only).
*Pre-configured scrape targets: Prometheus self-monitoring, Node Exporter (system + ZFS metrics), and IPMI Exporter.


----
----


For further customization or troubleshooting, refer to the [https://prometheus.io/docs/ upstream Prometheus documentation] or contact Open-E support.
To view or troubleshoot the collected metrics, see the [[Extension:Grafana_rev_03|Grafana extension]] article or contact Open-E support.


[[Category:Help topics]]
[[Category:Help topics]]
[[Category:Extensions]]
[[Category:Extensions]]

Revision as of 10:08, 7 July 2026

Prometheus is a metrics database that works in the background as part of the storage server's monitoring feature. Once installed, it continuously collects system, storage, and hardware sensor data and makes that data available to the dashboards provided by the Grafana extension.

Note: Prometheus is delivered as an optional Small Update (the xc-prometheus module). This article describes revision 01. If your system was updated to a newer revision, refer to the matching Extension:Prometheus_rev_NN article.

How it works

Prometheus is a companion extension for the monitoring dashboards feature — it has no page of its own in the GUI. It works together with two other extensions:

  • The Metrics exporters extension, which gathers system, storage, and hardware sensor data on the storage server.
  • The Grafana extension, which reads the collected data from Prometheus and displays it as dashboards in your browser.

All three extensions — Metrics exporters, Prometheus, and Grafana — must be installed for the dashboards to display data.

Once installed, Prometheus runs automatically in the background. There is nothing to configure, and no separate login or user interface — all interaction with the collected metrics happens through the Grafana dashboards.

To view the metrics that Prometheus collects, open the dashboards provided by the Grafana extension.

Known issues and limitations

  • All companion extensions must be installed. Prometheus has no data to collect without the Metrics exporters extension, and no dashboards can display its data without the Grafana extension. If any of the three extensions is missing, the dashboards will show No data for all panels.

Storage and data retention

The metrics database automatically keeps roughly the last 15 days of measurements; older data is discarded automatically, so there is nothing to clean up.

It uses a dedicated storage area of about 1 GB. Under normal operation, 15 days of metrics use only a fraction of that space, so the database does not fill up in typical use.

If the storage area ever fills — for example after an unusually high volume of metrics — the database temporarily pauses recording new measurements until space is automatically reclaimed as the oldest data ages out. No user action is required; it recovers on its own. The only visible effect is a gap in the dashboards for the affected period, and existing data and other services are unaffected.

Changelog

rev 01

  • Initial release of the Prometheus metrics-database extension.

To view or troubleshoot the collected metrics, see the Grafana extension article or contact Open-E support.