License
MIT License
The Sensu PagerDuty Handler is a Sensu Event Handler which manages
PagerDuty incidents, for alerting operators. With this handler,
Sensu can trigger and resolve PagerDuty incidents.
The Sensu Go PagerDuty handler for incident management
Usage:
sensu-pagerduty-handler [flags]
sensu-pagerduty-handler [command]
Available Commands:
completion Generate the autocompletion script for the specified shell
help Help about any command
version Print the version number of this plugin
Flags:
-e, --alternate-endpoint string The endpoint to use to send the PagerDuty events, can be set with PAGERDUTY_ALTERNATE_ENDPOINT
--contact-routing Enable contact routing
-k, --dedup-key-template string The PagerDuty V2 API deduplication key template, can be set with PAGERDUTY_DEDUP_KEY_TEMPLATE (default "{{.Entity.Name}}-{{.Check.Name}}")
--details-format string The format of the details output ('string' or 'json'), can be set with PAGERDUTY_DETAILS_FORMAT (default "string")
-d, --details-template string The template for the alert details, can be set with PAGERDUTY_DETAILS_TEMPLATE (default full event JSON)
-h, --help help for sensu-pagerduty-handler
-s, --status-map string The status map used to translate a Sensu check status to a PagerDuty severity, can be set with PAGERDUTY_STATUS_MAP
-S, --summary-template string The template for the alert summary, can be set with PAGERDUTY_SUMMARY_TEMPLATE (default "{{.Entity.Name}}/{{.Check.Name}} : {{.Check.Output}}")
--team string Envvar name for pager team(alphanumeric and underscores) holding PagerDuty V2 API authentication token, can be set with PAGERDUTY_TEAM
--team-suffix string Pager team suffix string to append if missing from team name, can be set with PAGERDUTY_TEAM_SUFFIX (default "_pagerduty_token")
--timeout uint The maximum amount of time in seconds to wait for the event to be created, can be set with PAGERDUTY_TIMEOUT (default 30)
-t, --token string The PagerDuty V2 API authentication token, can be set with PAGERDUTY_TOKEN
Use "sensu-pagerduty-handler [command] --help" for more information about a command.
The deduplication key is determined via the --dedup-key-template
argument. It
is a Golang template containing the event values and defaults to
{{.Entity.Name}}-{{.Check.Name}}
.
Optionally you can provide mapping information between the Sensu check status
and the PagerDuty incident severity. To provide the mapping you need to use the
--status-map
command line option or the PAGERDUTY_STATUS_MAP
environment
variable. The option accepts a JSON document containing the mapping
information. Here's an example of the JSON document:
{
"info": [
0,
1
],
"warning": [
2
],
"critical:": [
3
],
"error": [
4,
5,
6,
7,
8,
9,
10
]
}
The valid PagerDuty alert severity levels are the following:
info
warning
critical
error
Sensu Assets are the best way to make use of this plugin. If you're not
using an asset, please consider doing so! If you're using sensuctl 5.13 with
Sensu Backend 5.13 or later, you can use the following command to add the asset:
sensuctl asset add sensu/sensu-pagerduty-handler
If you're using an earlier version of sensuctl, you can find the asset on the
Bonsai Asset Index.
---
type: Handler
api_version: core/v2
metadata:
name: pagerduty
namespace: default
spec:
type: pipe
command: >-
sensu-pagerduty-handler
--dedup-key-template "{{.Entity.Namespace}}-{{.Entity.Name}}-{{.Check.Name}}"
--status-map "{\"info\":[0],\"warning\": [1],\"critical\": [2],\"error\": [3,127]}"
--summary-template "[{{.Entity.Namespace}}] {{.Entity.Name}}/{{.Check.Name}}: {{.Check.State}}"
--details-template "{{.Check.Output}}\n\n{{.Check}}"
--details-format string
timeout: 10
runtime_assets:
- sensu/sensu-pagerduty-handler
filters:
- is_incident
secrets:
- name: PAGERDUTY_TOKEN
secret: pagerduty_authtoken
Most arguments for this handler are available to be set via environment
variables. However, any arguments specified directly on the command line
override the corresponding environment variable.
Argument | Environment Variable |
---|---|
--alternate-endpoint | PAGERDUTY_ALTERNATE_ENDPOINT |
--dedup-key-template | PAGERDUTY_DEDUP_KEY_TEMPLATE |
--details-template | PAGERDUTY_DETAILS_TEMPLATE |
--details-format | PAGERDUTY_DETAILS_FORMAT |
--status-map | PAGERDUTY_STATUS_MAP |
--summary-template | PAGERDUTY_SUMMARY_TEMPLATE |
--team | PAGERDUTY_TEAM |
--team-suffix | PAGERDUTY_TEAM_SUFFIX |
--timeout | PAGERDUTY_TIMEOUT |
--token | PAGERDUTY_TOKEN |
Security Note: Care should be taken to not expose the auth token for this
handler by specifying it on the command line or by directly setting the
environment variable in the handler definition. It is suggested to make use of
secrets management to surface it as an environment variable. The handler
definition above references it as a secret. Below is an example secrets
definition that make use of the built-in env secrets provider.
---
type: Secret
api_version: secrets/v1
metadata:
name: pagerduty_token
spec:
provider: env
id: PAGERDUTY_TOKEN
This handler provides options for using templates to populate the values
provided by the event in the message sent via SNS. More information on
template syntax and format can be found in the documentation.
All arguments for this handler are tunable on a per entity or check basis based
on annotations. The annotations keyspace for this handler is
sensu.io/plugins/sensu-pagerduty-handler/config
.
NOTE: Due to check token substituion, supplying a template value such
as for details-template
as a check annotation requires that you place the
desired template as a golang string literal (enlcosed in backticks)
within another template definition. This does not apply to entity annotations.
To change the --details-template
argument for a particular check, and taking
into account the note above regarding templates, for that check's metadata add
the following:
type: CheckConfig
api_version: core/v2
metadata:
annotations:
sensu.io/plugins/sensu-pagerduty-handler/config/details-template: "{{`{{.Check.Output}}`}}"
[ ... ]
To change the --token
argument for a particular check, for that checks's metadata
add the following:
type: CheckConfig
api_version: core/v2
metadata:
annotations:
sensu.io/plugins/sensu-pagerduty-handler/config/token: abcde12345fabcd67890efabc12345de
[ ... ]
Instead of specifying the authentication token directly in the check or agent annotations, you can instead reference a
pager team name, which will then be used to lookup the corresponding token from the handler environment.
Corresponding pager team token environment variables can be populated in the handler environment in 3 different ways
Pager team names will be automatically suffixed with configured --team-suffix (default: _pagerduty_suffix
)
Note: Pager team name strings should be alphameric and underscores only. Groups of illegal characters will be mapped
into a single underscore character. Ex: example-_-team
will be converted to example_team
If the team token lookup fails, the explicitly provided token will be used as a fallback if available.
First set the team annotation in the check or agent resource.
---
type: CheckConfig
api_version: core/v2
metadata:
name: example-check
annotations:
sensu.io/plugins/sensu-pagerduty-handler/config/team: team_1
And define the corresponding evironment variable for the pager team's token in the handler's environment.
---
type: Handler
api_version: core/v2
metadata:
name: pagerduty
namespace: default
spec:
type: pipe
command: >-
sensu-pagerduty-handler
--dedup-key-template "{{.Entity.Namespace}}-{{.Entity.Name}}-{{.Check.Name}}"
--status-map "{\"info\":[0],\"warning\": [1],\"critical\": [2],\"error\": [3,127]}"
--summary-template "[{{.Entity.Namespace}}] {{.Entity.Name}}/{{.Check.Name}}: {{.Check.State}}"
--details-template "{{.Check.Output}}\n\n{{.Check}}"
timeout: 10
runtime_assets:
- sensu/sensu-pagerduty-handler
filters:
- is_incident
secrets:
- name: PAGERDUTY_TOKEN
secret: pagerduty_authtoken
env_vars:
- team_1_pagerduty_token="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
The Sensu Pagerduty Handler provides support for generating one Pagerduty event per "contact" via
the --contact-routing
flag.
With --contact-routing
enabled, the Sensu Pagerduty Handler will do the following:
Check for and merge the entity, check, and/or event contacts
annotation.
The contacts
annotation supports a comma-separated list of contact names containing alpha-numeric characters and
underscore (_
) characters only.
Example:
annotations:
contacts: "team_a,team_b"
NOTE: when --contact-routing
is enabled, the handler will log a message
like Pagerduty contact routing is enabled (contacts: team_a, team_b)
.
If --contact-routing
is enabled and no contacts
annotations are found, the handler will log an error
like contact routing enabled but no contacts were found
.
Lookup contact-specific environment variables for Pagerduty API Authentication
When --contact-routing
is enabled, the Sensu Pagerduty Handler will attempt to create or update an event per
configured "contact".
For each configured "contact", the Sensu Pagerduty Handler will look for a matching environment variable containing a
Pagerduty token.
Pagerduty token environment variables should be prefixed with PAGERDUTY_TOKEN_
and match the contact name in
all-uppercase (e.g. the contact "team_a" needs a corresponding PAGERDUTY_TOKEN_TEAM_A
environment variable).
NOTE: contact names are converted to environment variables
via fmt.Sprintf("PAGERDUTY_TOKEN_%s",strings.ToUpper(contact))
.
If a matching contact environment variable is found, the event will be processed.
If the contact environment variable is not found, the handler will log a warning (
e.g. WARNING: skipping contact: "team-a" (no environment variable found for "PAGERDUTY_TOKEN_TEAM_A")
\n).
---
api_version: core/v2
type: Handler
metadata:
name: pagerduty
spec:
type: pipe
command: >-
sensu-pagerduty-handler
--contact-routing
--dedup-key-template "{{.Entity.Namespace}}-{{.Entity.Name}}-{{.Check.Name}}"
--status-map "{\"info\":[0],\"warning\": [1],\"critical\": [2],\"error\": [3,127]}"
--summary-template "[{{.Entity.Namespace}}] {{.Entity.Name}}/{{.Check.Name}}: {{.Check.State}}"
--details-template "{{ .Check.Name }} is {{ .Check.State }} on {{ .Entity.Name }} (namespace: {{ .Entity.Namespace }})"
timeout: 10
filters: [ ]
runtime_assets: [ ]
env_vars: [ ]
secrets:
- name: PAGERDUTY_TOKEN_TEAM_A
secret: pagerduty_token_team_a
- name: PAGERDUTY_TOKEN_TEAM_B
secret: pagerduty_token_team_b
NOTE: contact routing is compatible with Sensu Secrets or environment variables set via Handler env_vars
, but given
the sensitive nature of a Pagerduty API token, using secrets management is strongly encouraged.
This handler supports the use of the environment variables HTTP_PROXY,
HTTPS_PROXY, and NO_PROXY (or the lowercase versions thereof). HTTPS_PROXY takes
precedence over HTTP_PROXY for https requests. The environment values may be
either a complete URL or a "host[:port]", in which case the "http" scheme is
assumed.
Download the latest version of the sensu-pagerduty-handler from releases,
or create an executable from this source.
From the local path of the sensu-pagerduty-handler repository:
go build -o /usr/local/bin/sensu-pagerduty-handler
See https://github.com/sensu/sensu-go/blob/master/CONTRIBUTING.md
Please describe the reason for reporting this asset. Our moderators will be notified and will disable the asset if it is found to be inappropriate.
×