Sensu Rundeck Handler

Table of Contents

• Overview
• Enterprise plugin
• Usage examples
• Configuration
  • Asset registration
  • Handler definition
  • API URL
  • Job invocation
  • Node targeting
  • Environment variables
  • Annotations
    • Annotations for remediation actions
    • Annotations for arguments
  • Proxy support

Overview

The Sensu Rundeck Handler is a Sensu Handler for initiating
Rundeck Jobs for automated remediation. This handler requires
access to the Rundeck API Server.

Enterprise plugin

The Sensu Rundeck Handler is a Sensu plugin that requires a valid Sensu license
to run. Sensu Go >= 5.21 will add the SENSU_LICENSE_FILE environment
variable to the handler execution. To run the plugin independently of Sensu
(e.g. test/dev), you must set the environment variable:

SENSU_LICENSE_FILE=$(sensuctl license info --format json)

Usage examples

Sensu Rundeck Remediation Handler

Usage:
  sensu-rundeck-handler [flags]
  sensu-rundeck-handler [command]

Available Commands:
  help        Help about any command
  version     Print the version number of this plugin

Flags:
  -a, --auth-token string        Authentication token for accssing the Rundeck API
  -u, --api-url string           URL for the Rundeck API (default "http://locallhost:4400/api/35")
  -n, --node-template string     The template to use for naming the node to be targeted (default "{{.Entity.Name}}")
  -t, --trusted-ca-file string   TLS CA certificate bundle in PEM format
  -i, --insecure-skip-verify     Skip TLS certificate verification (not recommended!)
  -h, --help                     help for sensu-rundeck-handler

Use "sensu-rundeck-handler [command] --help" for more information about a command.

Configuration

Asset registration

Sensu Assets are the best way to make use of this plugin. If you're not
using an asset, please consider doing so! If you're using sensuctl 5.13 with
Sensu Backend 5.13 or later, you can use the following command to add the asset:

sensuctl asset add sensu/sensu-rundeck-handler

If you're using an earlier version of sensuctl, you can find the asset on the
Bonsai Asset Index.

Handler definition

---
type: Handler
api_version: core/v2
metadata:
  name: sensu-rundeck-handler
  namespace: default
spec:
  command: sensu-rundeck-handler  --api-url http://rundeck.example.com:4440/
  type: pipe
  runtime_assets:
    - sensu/sensu-rundeck-handler
  filters:
    - not_silenced
  secrets:
    - name: RUNDECK_API_KEY
      secret: rundeck_api_key

API URL

Rundeck versions their API by including the version number as part of the
URL. This handler makes use of version 33. The --api-url argument provided
should not include the /api path or this version number. You only need to
provide the URL to the root path. The default is http://localhost:4440/.

Job invocation

Rundeck offers two options for running jobs. The first is via the standard
job which requires authentication using a token and uses a
node filter submitted with the job to target the entity from the event.

The second is via a webhook which does not require authentication and,
optionally, supports targeting the entity from the event as part of the
webhook definition in Rundeck. This operation requires the webhook name
(e.g. 8TlVUt1CQcbD3K5fiGGKNR1zBg7lnZsS#RestartNGINX), not the full URL, be
provided.

Node targeting

For standard job invocation, the handler uses a template to determine which
value in the event data to use for node targeting. The default template is
{{.Entity.Name}}. In the event this does not match up with the node name in
Rundeck, the template is configurable using the --node-template argument.

For webhook job invocation, the full event JSON is sent to the webhook. In
order to target the node you will have to specify the event value to use in the
Node Filter specification for the webhook in Rundeck. To use the entity
similar to the standard job invocation above you would use
name: ${data.entity.metadata.name}.

Environment variables

Most arguments for this handler are available to be set via environment
variables. However, any arguments specified directly on the command line
override the corresponding environment variable.

Security Note: Care should be taken to not expose the auth token for this
handler by specifying it on the command line or by directly setting the
environment variable in the handler definition. It is suggested to make use of
secrets management to surface it as an environment variable. The handler
definition above references the auth token as a secret. Below is an example
secrets definition that make use of the built-in env secrets provider.

---
type: Secret
api_version: secrets/v1
metadata:
  name: rundeck_api_key
spec:
  provider: env
  id: RUNDECK_API_KEY

Annotations

The annotations keyspace for this handler is
sensu.io/plugins/sensu-rundeck-handler/config.

Annotations for Rundeck Jobs

To specify the remediation action(s) for a check, assign a JSON similar to the
following to the actions annotation in the handler's keyspace (noted above).

For a standard job:

[
  {
    "rundeck_job": "848c5953-77ab-4261-adfb-cacb33159d84",
    "occurrences": [ 3 ],
    "severities": [ 1, 2 ]
  }
]

For a webhook:

[
  {
    "webhook": "8TlVUt1CQcbD3K5fiGGKNR1zBg7lnZsS#RestartNGINX",
    "occurrences": [ 3 ],
    "severities": [ 1, 2 ]
  }
]

For example, if you have an NGINX web server running and you wanted to attempt
remediation by using a Rundeck job for restarting the nginx service on the the
third occurrence of the check returning 1 (warning) or 2 (critical), you would
specify it as so:

type: CheckConfig
api_version: core/v2
metadata:
  annotations:
    sensu.io/plugins/sensu-rundeck-handler/config/actions: |
      [
        {
          "rundeck_job": "848c5953-77ab-4261-adfb-cacb33159d84",
          "occurrences": [ 3 ],
          "severities": [ 1, 2 ]
        }
      ]
spec:
  handlers:
    - sensu-rundeck-handler

The actions can contain multiple standard jobs and/or webhooks to run on the
same or different combination of occurrences and severities. For example, if
you wanted to extend the configuration above by attempting a different Rundeck
job (e.g. one to re-deploy the webapp running under NGINX) if the job to
restart NGINX fails to rememediate and the occurrences then reaches 6, you would
specify the following:

type: CheckConfig
api_version: core/v2
metadata:
  annotations:
    sensu.io/plugins/sensu-rundeck-handler/config/actions: |
      [
        {
          "rundeck_job": "848c5953-77ab-4261-adfb-cacb33159d84",
          "occurrences": [ 3 ],
          "severities": [ 1, 2 ]
        },
        {
          "rundeck_job": "37044c9f-79e2-435e-bc03-db81995cc99e",
          "occurrences": [ 6 ],
          "severities": [ 1, 2 ]
        }
      ]
spec:
  handlers:
    - sensu-rundeck-handler

The same can be done with webhooks or you can mix and match standard jobs with
webhooks as seen below:

type: CheckConfig
api_version: core/v2
metadata:
  annotations:
    sensu.io/plugins/sensu-rundeck-handler/config/actions: |
      [
        {
          "webhook": "8TlVUt1CQcbD3K5fiGGKNR1zBg7lnZsS#RestartNGINX",
          "occurrences": [ 3 ],
          "severities": [ 1, 2 ]
        },
        {
          "rundeck_job": "37044c9f-79e2-435e-bc03-db81995cc99e",
          "occurrences": [ 6 ],
          "severities": [ 1, 2 ]
        }
      ]
spec:
  handlers:
    - sensu-rundeck-handler

Annotations for arguments

All arguments for this handler are tunable on a per entity or check basis based
on annotations.

Examples

To change the Rundeck API URL argument (--api-url) for a particular check,
in that checks's metadata add the following (building from the example above):

type: CheckConfig
api_version: core/v2
metadata:
  annotations:
    sensu.io/plugins/sensu-rundeck-handler/config/api-url: "https://rundeck-api.example.com:4440/"
    sensu.io/plugins/sensu-rundeck-handler/config/actions: |
      [
        {
          "rundeck_job": "848c5953-77ab-4261-adfb-cacb33159d84",
          "occurrences": [ 3 ],
          "severities": [ 1, 2 ]
        }
      ]
spec:
  handlers:
    - sensu-rundeck-handler
[...]

Proxy support

This handler supports the use of the environment variables HTTP_PROXY,
HTTPS_PROXY, and NO_PROXY (or the lowercase versions thereof). HTTPS_PROXY takes
precedence over HTTP_PROXY for https requests. The environment values may be
either a complete URL or a "host[:port]", in which case the "http" scheme is
assumed.