Prometheus rev 01

From Open-E Wiki
Jump to navigation Jump to search
Ai-B (talk | contribs)
Create Prometheus extension article rev 01 (xc-prometheus SU)
 
Ai-B (talk | contribs)
m Remove Category:Help topics - Extension articles are listed only under Category:Extensions
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
__NOTOC__
__NOTOC__
Prometheus is a metrics database packaged as an LXC container Small Update for the storage server. Once installed, it continuously collects system, ZFS storage, and hardware sensor data from the appliance and makes it available for querying and visualisation.
Prometheus is a metrics database that works in the background as part of the storage server's monitoring feature. Once installed, it continuously collects system, storage, and hardware sensor data and makes that data available to the dashboards provided by the Grafana extension.


'''Note:''' Prometheus is delivered as an optional Small Update (the '''xc-prometheus''' module). This article describes revision '''01'''. If your system was updated to a newer revision, refer to the matching ''Extension:Prometheus_rev_NN'' article.
'''Note:''' Prometheus is delivered as an optional Small Update (the '''xc-prometheus''' module). This article describes revision '''01'''. If your system was updated to a newer revision, refer to the matching ''Extension:Prometheus_rev_NN'' article.
Line 6: Line 6:
== How it works ==
== How it works ==


Prometheus scrapes metrics from exporters running on the storage server (provided by the '''x-metrics-exp''' companion module) and stores them in its time-series database (TSDB). The Grafana extension ('''xc-grafana''') then reads from Prometheus to render dashboards in the browser.
Prometheus is a companion extension for the monitoring dashboards feature — it has no page of its own in the GUI. It works together with two other extensions:


The full data flow is:
*The [[Extension:Metrics_Exporters_rev_03|Metrics exporters]] extension, which gathers system, storage, and hardware sensor data on the storage server.
*The [[Extension:Grafana_rev_03|Grafana]] extension, which reads the collected data from Prometheus and displays it as dashboards in your browser.


*The storage server runs a Node Exporter (system metrics: CPU, memory, disk, and network, plus ZFS pool metrics via a textfile collector) and an IPMI Exporter (hardware sensor metrics).
All three extensions — [[Extension:Metrics_Exporters_rev_03|Metrics exporters]], Prometheus, and [[Extension:Grafana_rev_03|Grafana]] — must be installed for the dashboards to display data.
*Prometheus scrapes the exporters every 15 seconds and writes the samples to its local TSDB.
*Grafana reads from Prometheus and renders the dashboards in the browser.


All three modules ('''x-metrics-exp''', '''xc-prometheus''', and '''xc-grafana''') must be installed for the dashboards to display data.
Once installed, Prometheus runs automatically in the background. There is nothing to configure, and no separate login or user interface — all interaction with the collected metrics happens through the Grafana dashboards. Measurements are collected every '''15 seconds''', which determines how fine-grained the data shown in the dashboards is.


== Scrape targets ==
To view the metrics that Prometheus collects, open the dashboards provided by the [[Extension:Grafana_rev_03|Grafana extension]].


Prometheus is pre-configured with three scrape jobs:
== Network access ==


{| class="wikitable"
Prometheus is available only on the storage server itself. It provides its web interface and query API on port '''9090''', bound to the local host ('''127.0.0.1'''), and is '''not''' reachable from the network. Only local services running on the same storage server — such as the Grafana extension — can access it, so the collected monitoring data is never exposed outside the appliance.
! Job !! Target !! Description
|-
| <code>prometheus</code> || <code>localhost:9090</code> || Prometheus self-monitoring (scrape interval: 5 s)
|-
| <code>node</code> || <code>localhost:9100</code>, <code>localhost:9290</code> || Node Exporter — system metrics (CPU, memory, disk, network) and ZFS pool metrics via textfile collector
|-
| <code>ipmi</code> || <code>localhost:9290</code> || IPMI Exporter — hardware sensor readings (temperatures, fan speeds, voltages, power)
|}


== Reference ==
== Known issues and limitations ==
 
{| class="wikitable"
! Item !! Value
|-
| Web UI / API address || <code>127.0.0.1:9090</code> (loopback only — not externally accessible)
|-
| Protocol || HTTP
|-
| TSDB data directory (inside container) || <code>/var/lib/prometheus</code>
|-
| Prometheus configuration (inside container) || <code>/etc/prometheus/prometheus.yml</code>
|-
| Scrape interval || 15 s (5 s for self-monitoring)
|-
| Memory limit || 1024 MB
|}
 
== Maintenance ==
 
=== Checking service status ===
 
From the storage server shell, check whether the Prometheus container is running:
 
  ssh localhost lxc-ls -f


The '''prometheus''' container should show state '''RUNNING'''.
*'''All companion extensions must be installed.''' Prometheus has no data to collect without the [[Extension:Metrics_Exporters_rev_03|Metrics exporters]] extension, and no dashboards can display its data without the [[Extension:Grafana_rev_03|Grafana]] extension. If any of the three extensions is missing, the dashboards will show ''No data'' for all panels.


=== Viewing logs ===
== Storage and data retention ==


Connect to the container console and inspect the Prometheus service log:
The metrics database automatically keeps roughly the last '''15 days''' of measurements; older data is discarded automatically, so there is nothing to clean up.


  journalctl -u prometheus --no-pager -n 50
It uses a dedicated storage area of about '''1&nbsp;GB'''. Under normal operation, 15 days of metrics use only a fraction of that space, so the database does not fill up in typical use.
 
=== Updates ===
 
Do not run system package updates inside the container. Updates are delivered as new container revisions through the standard small-update mechanism.
 
== Known issues and limitations ==


*'''All three companion modules must be installed.''' Prometheus requires the '''x-metrics-exp''' Small Update to have any metrics to scrape. Grafana ('''xc-grafana''') requires Prometheus to be running to display dashboards. If any module is missing, dashboards will show ''No data'' for all panels.
If the storage area ever fills — for example after an unusually high volume of metrics — the database temporarily pauses recording new measurements until space is automatically reclaimed as the oldest data ages out. No user action is required; it recovers on its own. The only visible effect is a gap in the dashboards for the affected period, and existing data and other services are unaffected.
*'''Web UI and API are loopback-only.''' The Prometheus web interface and PromQL API are bound to <code>127.0.0.1:9090</code> and are not accessible from external hosts. This is intentional — Prometheus is a backend service accessed only by Grafana and for local troubleshooting from the storage server shell.
*'''No in-container package updates.''' Running <code>apt upgrade</code> or similar inside the container is not supported due to disk space constraints. Updates are delivered as new container revisions.


== Changelog ==
== Changelog ==
Line 78: Line 37:
=== rev 01 ===
=== rev 01 ===


*Initial release: Prometheus packaged as a standalone LXC container Small Update ('''xc-prometheus''' module), separated from the combined Grafana/Prometheus stack.
*Initial release of the Prometheus metrics-database extension.
*Prometheus web UI and API bound to <code>127.0.0.1:9090</code> (loopback only).
*Pre-configured scrape targets: Prometheus self-monitoring, Node Exporter (system + ZFS metrics), and IPMI Exporter.


----
----


For further customization or troubleshooting, refer to the [https://prometheus.io/docs/ upstream Prometheus documentation] or contact Open-E support.
To view or troubleshoot the collected metrics, see the [[Extension:Grafana_rev_03|Grafana extension]] article or contact Open-E support.


[[Category:Help topics]]
[[Category:Extensions]]
[[Category:Extensions]]

Latest revision as of 10:31, 13 October 2026

Prometheus is a metrics database that works in the background as part of the storage server's monitoring feature. Once installed, it continuously collects system, storage, and hardware sensor data and makes that data available to the dashboards provided by the Grafana extension.

Note: Prometheus is delivered as an optional Small Update (the xc-prometheus module). This article describes revision 01. If your system was updated to a newer revision, refer to the matching Extension:Prometheus_rev_NN article.

How it works

Prometheus is a companion extension for the monitoring dashboards feature — it has no page of its own in the GUI. It works together with two other extensions:

  • The Metrics exporters extension, which gathers system, storage, and hardware sensor data on the storage server.
  • The Grafana extension, which reads the collected data from Prometheus and displays it as dashboards in your browser.

All three extensions — Metrics exporters, Prometheus, and Grafana — must be installed for the dashboards to display data.

Once installed, Prometheus runs automatically in the background. There is nothing to configure, and no separate login or user interface — all interaction with the collected metrics happens through the Grafana dashboards. Measurements are collected every 15 seconds, which determines how fine-grained the data shown in the dashboards is.

To view the metrics that Prometheus collects, open the dashboards provided by the Grafana extension.

Network access

Prometheus is available only on the storage server itself. It provides its web interface and query API on port 9090, bound to the local host (127.0.0.1), and is not reachable from the network. Only local services running on the same storage server — such as the Grafana extension — can access it, so the collected monitoring data is never exposed outside the appliance.

Known issues and limitations

  • All companion extensions must be installed. Prometheus has no data to collect without the Metrics exporters extension, and no dashboards can display its data without the Grafana extension. If any of the three extensions is missing, the dashboards will show No data for all panels.

Storage and data retention

The metrics database automatically keeps roughly the last 15 days of measurements; older data is discarded automatically, so there is nothing to clean up.

It uses a dedicated storage area of about 1 GB. Under normal operation, 15 days of metrics use only a fraction of that space, so the database does not fill up in typical use.

If the storage area ever fills — for example after an unusually high volume of metrics — the database temporarily pauses recording new measurements until space is automatically reclaimed as the oldest data ages out. No user action is required; it recovers on its own. The only visible effect is a gap in the dashboards for the affected period, and existing data and other services are unaffected.

Changelog

rev 01

  • Initial release of the Prometheus metrics-database extension.

To view or troubleshoot the collected metrics, see the Grafana extension article or contact Open-E support.