Skip to content

Commit

Permalink
Updated README for reminder and profiles and fixed a couple bugs
Browse files Browse the repository at this point in the history
  • Loading branch information
fusiondog committed Jan 8, 2016
1 parent 97500e7 commit 87cd594
Show file tree
Hide file tree
Showing 4 changed files with 27 additions and 23 deletions.
15 changes: 12 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ consul-alerts

[![Join the chat at https://gitter.im/AcalephStorage/consul-alerts](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/AcalephStorage/consul-alerts?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

A simple daemon to send notifications based on Consul health checks.
A highly available daemon to send notifications and reminders based on Consul health checks. Including profile selection based on service, check, or host that enables specific handlers and reminder intervals.

## Requirement

Expand Down Expand Up @@ -84,6 +84,15 @@ To prevent flapping, notifications are only sent when a check status has been st

eg. `consul-alerts/config/checks/change-threshold` = `30`

#### Enable Profiles Selection
Profiles may be configured as keys in consul-alerts/config/notif-profiles/. The key name is the name of the profile and the value should be a JSON object with an "Interval" key set to an int in minutes and a key "NotifList" that should be an object of profile names as keys and true for the value.

Profile selection is done by setting keys in consul-alerts/config/notif-selection/services/, consul-alerts/config/notif-selection/checks/, or consul-alerts/config/notif-selection/hosts/ with the appropriate service, check, or host name as the key and the selected profile name as the value.

Reminders are processed every five minutes. Interval values should be a multiple of five. If the Interval value is 0 or not set then reminders will not be set.

The default profile may be set as the fallback to any checks that do not match a selection. If there is no default profile set then the full list of enabled notifiers will be used and no reminders.

#### Enable/Disable Specific Health Checks

There are four ways to enable/disable health check notifications: mark them by node, serviceID, checkID, or mark individually by node/serviceID/checkID. This is done by adding a KV entry in `consul-alerts/config/checks/blacklist/...`. Removing the entry will re-enable the check notifications.
Expand Down Expand Up @@ -112,7 +121,7 @@ Handlers can be configured by adding them to `consul-alerts/config/events/handle

### Notifiers

There are four builtin notifiers. Only the *Log* notifier is enabled by default. It is also possible to add custom notifiers similar to custom event handlers. Custom notifiers can be added in `consul-alerts/config/notifiers/custom`.
There are four builtin notifiers. Only the *Log* notifier is enabled by default. It is also possible to add custom notifiers similar to custom event handlers. Custom notifiers can be added as keys with command path string values in `consul-alerts/config/notifiers/custom/`. The keys will be used as notifier names in the profiles.

#### Logger

Expand Down Expand Up @@ -247,4 +256,4 @@ PRs are more than welcome. Just fork, create a feature branch, and open a PR. We
TODO
----

This is a port from a tool we developed recently, there are still a few things missing like loading a custom configuration via command/api instead of manually editing consul's KV. Also need to set up a reminder feature. Needs better doc and some cleanup too. :)
Needs better doc and some cleanup too. :)
30 changes: 12 additions & 18 deletions consul/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -485,36 +485,30 @@ func (c *ConsulAlertClient) CheckStatus(node, serviceId, checkId string) (status
}

func (c *ConsulAlertClient) GetProfileInfo(node, serviceId, checkId string) (notifiersList map[string]bool, interval int) {
key := fmt.Sprintf("consul-alerts/config/notif-selection/services/%s", serviceId)
kvPair, _, _ := c.api.KV().Get(key, nil)
log.Println("Getting profile for node: ", node, " service: ", serviceId, " check: ", checkId)

var profile string

if kvPair == nil {
log.Println("service selection key not found.")
key = fmt.Sprintf("consul-alerts/config/notif-selection/check/%s", checkId)
kvPair, _, _ = c.api.KV().Get(key, nil)
kvPair, _, _ := c.api.KV().Get(fmt.Sprintf("consul-alerts/config/notif-selection/services/%s", serviceId), nil)
if kvPair != nil {
profile = string(kvPair.Value)
if kvPair == nil {
log.Println("check selection key not found.")
key = fmt.Sprintf("consul-alerts/config/notif-selection/host/%s", checkId)
kvPair, _, _ = c.api.KV().Get(key, nil)
profile = string(kvPair.Value)
if kvPair == nil {
log.Println("no selection key found.")
profile = "default"
}
}
} else {
log.Println("service selection key found.")
} else if kvPair, _, _ = c.api.KV().Get(fmt.Sprintf("consul-alerts/config/notif-selection/checks/%s", checkId), nil); kvPair != nil {
profile = string(kvPair.Value)
log.Println("check selection key found.")
} else if kvPair, _, _ = c.api.KV().Get(fmt.Sprintf("consul-alerts/config/notif-selection/hosts/%s", node), nil); kvPair != nil {
profile = string(kvPair.Value)
log.Println("host selection key found.")
} else {
profile = "default"
}

key = fmt.Sprintf("consul-alerts/config/notif-profiles/%s", profile)
key := fmt.Sprintf("consul-alerts/config/notif-profiles/%s", profile)
log.Println("profile key: ", key)
kvPair, _, _ = c.api.KV().Get(key, nil)
if kvPair == nil {
log.Println("profile key not found.")
return
}
var checkProfile ProfileInfo
json.Unmarshal(kvPair.Value, &checkProfile)
Expand Down
1 change: 1 addition & 0 deletions notifier/email-notifier.go
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ func (emailNotifier *EmailNotifier) Notify(alerts Messages) bool {
log.Println("Unable to send notification:", err)
return false
}
log.Println("Email notification sent.")
return true
}

Expand Down
4 changes: 2 additions & 2 deletions send-notifs.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ func (n *NotifEngine) sendBuiltin(messages notifier.Messages) {
filteredMessages := make(notifier.Messages, 0)
notifName := n.NotifierName()
for _, m := range messages {
if _, exists := m.NotifList[notifName]; exists {
if boolVal, exists := m.NotifList[notifName]; ( exists && boolVal ) || len(m.NotifList) == 0 {
filteredMessages = append(filteredMessages, m)
}
}
Expand All @@ -65,7 +65,7 @@ func (n *NotifEngine) sendCustom(messages notifier.Messages) {
for notifName, notifCmd := range consulClient.CustomNotifiers() {
filteredMessages := make(notifier.Messages, 0)
for _, m := range messages {
if _, exists := m.NotifList[notifName]; exists {
if boolVal, exists := m.NotifList[notifName]; ( exists && boolVal ) || len(m.NotifList) == 0 {
filteredMessages = append(filteredMessages, m)
}
}
Expand Down

0 comments on commit 87cd594

Please sign in to comment.