Sensu Go Graphite Handler Plugin

• Overview
• Usage examples
• Configuration
  • Sensu Go
    • Asset registration
    • Asset definition
    • Check definition
    • Handler definition
  • Sensu Core
• Installation from source
• Additional notes
• Contributing

Overview

The Sensu Graphite Handler is a Sensu Event Handler that sends metrics to the time series database Graphite. Sensu can collect metrics using check output metric extraction. Those collected metrics pass through the event pipeline, allowing Sensu to deliver the metrics to the configured metric event handlers. This Graphite handler will allow you to store, instrument, and visualize the metric data from Sensu.

This handler have been copied from nixwiz and from kuleuven.

Why another graphite handler:

In the original repository from nixwiz, each metric is appended by the entity name and the entity check name.

myprefix.myhost_domain.com.my-check-name_vmstat.nr_free_pages

As this is not what we expect, we modify this handler to only prefix the metric and take the metric name, like this:

myprefix.vmstat.nr_free_pages

Main goals:

• Add prefix to the metric name
• Add support to append entity labels to the prefix
• Add support to append entity annotations to the prefix

Usage examples

Help

Usage:
  sensu-go-graphite-handler [flags]
  sensu-go-graphite-handler [command]

Available Commands:
  help        Help about any command
  version     Print the version number of this plugin

Flags:
  -h, --help                  help for sensu-go-graphite-handler
  -l, --labels string         the annotations names (comma-separated) of entity annotations that must be added to the prefix in graphite for these metrics
  -a, --annotations string    the labels names (comma-separated) of entity labels that must be added to the prefix in graphite for these metrics
  -H, --host string           the hostname or address of the graphite server (default "127.0.0.1")
  -p, --port uint             the port number to which to connect on the graphite server (default 2003)
  -n, --no-prefix             unsets the default prefix value, use the bare metrics.point.name

Example: use entity annotation for metric prefix

1/ First configure the annotations on your sensu agent.

In /etc/sensu/agent.yaml

[...]
annotations:
  short_name: myhost
  short_dc: dc1

2/ Create the graphite handler

{
    "api_version": "core/v2",
    "type": "Handler",
    "metadata": {
        "namespace": "default",
        "name": "graphite"
    },
    "spec": {
        "type": "pipe",
        "command": "sensu-go-graphite-handler --prefix myprefix --annotations short_dc -H my-graphite-server.test.com",
        "timeout": 10,
        "filters": [
            "has_metrics"
        ]
    }
}

3/ In the check definition

{
    "api_version": "core/v2",
    "type": "CheckConfig",
    "metadata": {
        "namespace": "default",
        "name": "metrics-memory-vmstat"
    },
    "spec": {
        "command": "metrics-memory-vmstat.rb --scheme {{.annotations.short_name}}.vmstat",
        "subscriptions":[
            "dummy"
        ],
        "publish": true,
        "interval": 10,
        "output_metric_format": "graphite_plaintext",
        "output_metric_handlers": [
            "graphite"
        ]
    }
}

4/ Metric output:

myprefix.dc1.myhost.vmstat.pgalloc_normal 0.000000 1585554685
myprefix.dc1.myhost.vmstat.pgalloc_movable 0.000000 1585554685
myprefix.dc1.myhost.vmstat.pgfree 3971198058.000000 1585554685
[...]

Configuration

Sensu Go

Asset registration

Assets are the best way to make use of this plugin. If you're not using an asset, please consider doing so! If you're using sensuctl 5.13 or later, you can use the following command to add the asset:

sensuctl asset add fgouteroux/sensu-go-graphite-handler

If you're using an earlier version of sensuctl, you can download the asset definition from this project's Bonsai asset index page or releases or create an executable script from this source.

From the local path of the sensu-go-graphite-handler repository:

go build -o /usr/local/bin/sensu-go-graphite-handler main.go

Asset definition

---
type: Asset
api_version: core/v2
metadata:
  name: sensu-go-graphite-handler
spec:
  url: https://assets.bonsai.sensu.io/793026667633e5cb3e60ba1d063eb5a38ac9cd6b/sensu-go-graphite-handler_0.1.0_linux_amd64.tar.gz
  sha512: af738d13865fdce508fc0c4457ef7473c01639cc92da98590d842eb535db0b51bccdef5c310adf0135b5e3b3677487fe7a1b4370ae3028367bc8117c3fb1824c

Check definition

{
    "api_version": "core/v2",
    "type": "CheckConfig",
    "metadata": {
        "namespace": "default",
        "name": "metrics-memory-vmstat"
    },
    "spec": {
        "command": "metrics-memory-vmstat.rb --scheme core.{{.annotations.short_name}}.vmstat",
        "subscriptions":[
            "dummy"
        ],
        "publish": true,
        "interval": 10,
        "output_metric_format": "graphite_plaintext",
        "output_metric_handlers": [
            "graphite"
        ]
    }
}

Handler definition

{
    "api_version": "core/v2",
    "type": "Handler",
    "metadata": {
        "namespace": "default",
        "name": "graphite"
    },
    "spec": {
        "type": "pipe",
        "command": "sensu-go-graphite-handler -H my-graphite-server.test.com",
        "timeout": 10,
        "filters": [
            "has_metrics"
        ]
    }
}

That's right, you can collect different types of metrics (ex. Influx, Graphite, OpenTSDB, Nagios, etc.), Sensu will extract and transform them, and this handler will populate them into your Graphite.

Sensu Core

N/A

Installation from source

Sensu Go

See the instructions above for asset registration.

Sensu Core

Install and setup plugins on Sensu Core.

Additional notes

N/A

Contributing

N/A