sensu-keepalived-check

Table of Contents

• Overview
• Usage examples
• Configuration
  • Asset registration
  • Check definition
• Installation from source
• Additional notes
• Contributing

Overview

The sensu-keepalived-check is a Sensu Check that queries keepalived over the system
D-Bus and reports the current state of a VRRP instance. The check goes critical when the instance
is not in the expected state (for example, a node expected to be Master that has fallen back to
Backup or Fault), which is useful for catching failovers and split-brain situations.

Keepalived must be started with its D-Bus interface enabled (--dbus / enable_dbus in the
global configuration) for this check to work.

Usage examples

Usage:
  sensu-keepalived-check [flags]

Flags:
  -h, --help               help for sensu-keepalived-check
      --interface string   Interface of the VRRP instance
      --router-id string   Virtual router id of the VRRP instance
      --state string       Expected (OK) state of the VRRP instance (default "Master")
      --data-file string   Path of the data file written by keepalived PrintData (default "/tmp/keepalived.data")
      --family string      Address family of the VRRP instance (IPv4 or IPv6) (default "IPv4")
      --warning int        Seconds in an unexpected state before WARNING (default 0: immediately)
      --critical int       Seconds in an unexpected state before CRITICAL (default 0: immediately)

• --interface and --router-id identify the VRRP instance and are required. They correspond
  to the interface and virtual router id configured in your vrrp_instance, and together with
  --family form the D-Bus object path
  /org/keepalived/Vrrp1/Instance/<interface>/<router-id>/<family>.
• --family selects the address family of the VRRP instance, IPv4 (default) or IPv6.
• --data-file is the path of the state dump keepalived writes when asked for the last
  transition timestamp. Only change it if your keepalived writes its data file somewhere
  other than /tmp/keepalived.data.
• --state is the state the instance is expected to be in: Master, Backup, or Fault.
  The comparison is case-insensitive but must match the full state name. Defaults to Master.
• --warning and --critical grant a grace period before an unexpected state alerts: the
  check stays OK until the instance has been in the unexpected state for --warning seconds,
  reports WARNING until --critical seconds, and CRITICAL after that. With the defaults (0)
  an unexpected state is CRITICAL immediately. If the last transition timestamp cannot be
  read, the time in state is unknown and an unexpected state escalates to CRITICAL right away.

Configuration

Asset registration

Sensu Assets are the best way to make use of this plugin. If you're not using an asset, please
consider doing so! If you're using sensuctl 5.13 with Sensu Backend 5.13 or later, you can use the
following command to add the asset:

sensuctl asset add elfranne/sensu-keepalived-check

If you're using an earlier version of sensuctl, you can find the asset on the Bonsai Asset Index.

Check definition

---
type: CheckConfig
api_version: core/v2
metadata:
  name: sensu-keepalived-check
  namespace: default
spec:
  command: sensu-keepalived-check --interface ens256 --router-id 61 --state Master
  output_metric_format: nagios_perfdata
  subscriptions:
  - keepalived
  runtime_assets:
  - elfranne/sensu-keepalived-check

The check emits a single nagios perfdata metric after the status summary:

• keepalived_bad_state_seconds — how long the instance has been in a state other than
  --state; 0 while it matches --state. It is derived from the last transition
  timestamp, so it is omitted when that cannot be read (for example, an unreadable data
  file), in which case the check prints just the status summary with no metric.

The check result itself — the OK/WARNING/CRITICAL status and the process exit code —
is unchanged; it is simply not emitted as a metric.

Setting output_metric_format: nagios_perfdata makes Sensu extract
keepalived_bad_state_seconds as a metric; add output_metric_handlers to forward it to
your metrics pipeline.

Dashboard (Grafana + InfluxDB 1.x / InfluxQL)

With the sensu-influxdb-handler, keepalived_bad_state_seconds becomes a measurement
of the same name, with the value in a value field and the reporting host in a
sensu_entity_name tag (the --legacy handler flag uses host instead — adjust the
query accordingly).

How long a node has been in a bad state — reads 0 while the instance is in its
expected --state and climbs while it is in any other state:

SELECT last("value") FROM "keepalived_bad_state_seconds"
WHERE $timeFilter
GROUP BY time($__interval), "sensu_entity_name" fill(null)

The check emits the ready-to-graph value directly, so the panel needs no now() — which
InfluxQL 1.x cannot use in a SELECT expression anyway; no Flux / InfluxDB 2.x required.

Installation from source

The preferred way of installing and deploying this plugin is to use it as an Asset. If you would
like to compile and install the plugin from source or contribute to it, download the latest version
or create an executable script from this source.

From the local path of the sensu-keepalived-check repository:

go build

Additional notes

The check connects to the system D-Bus, so it must run on the same host as the keepalived
instance being monitored (typically with sufficient privileges to reach the
org.keepalived.Vrrp1 bus name).

Contributing

For more information about contributing to this plugin, see Contributing.