fixes

- improved log levels
- less error prone float string parsing
- TODOs for more & better measurement_faulty checks
- go fmt

README.md

@@ -2,79 +2,50 @@
 
 Cross platform command line application to run health checks against sensor stations registered on [openSenseMap.org](https://opensensemap.org).
 
-This tool quickly runs various health checks on selected senseBoxes,
-and can send out notifications via various protocols.
-Specifically email is implemented, but other transports can be added easily.
+This tool lets you automatically check if senseBoxes are still runnning correctly,
+and when that's not the case, notifies you.
+Currently, email notifications are implemented, but other transports can be added easily.
+Implemented health checks are [described below](#available-healthchecks), and new ones can be added just as easily (given some knowledge of programming).
 
-The tool can also operate in watch mode, checking boxes at regular intervals.
+The tool has multiple modes of operation:
 
-Check the manual in the [doc/](doc/osem_notify.md) directory for a description of features.
+- `osem_notify check boxes`: run one-off checks on boxes
+- `osem_notify watch boxes`: check boxes continuously.
+
+Run `osem_notify help` or check the manual in the [docs/](docs/osem_notify.md) directory for more details.
 
 ## get it
-Download a build from the [releases page](releases/).
-You can immediately call the application by running `./osem_notify` in a terminal in your downloads directory.
+Download a build from the [releases page](https://github.com/noerw/osem_notify/releases/).
+You can run the application by running `./osem_notify*` in a terminal in your downloads directory.
 
-On unix platforms you may add it to your path for convenience, so it is always callable via `osem_notify`:
+On unix platforms you may need to make it executable, and can add it to your `$PATH` for convenience, so it is always callable via `osem_notify`:
 ```sh
-sudo mv osem_notify /usr/bin/osem_notify
+chmod +x osem_notify*
+sudo mv osem_notify* /usr/bin/osem_notify
 ```
 
 ## configure it
-Configuration is required to set up notification transports, and can set the default healthchecks.
+The tool works out of the box for basic functionality, but must be configured to set up notifications.
 Configuration can be done via a YAML file located at `~/.osem_notify.yml` or through environment variables.
-Example configuration:
-
-```yaml
-defaultHealthchecks:
-  notifications:
-    transport: email
-    options:
-      recipients:
-      - fridolina@example.com
-      - ruth.less@example.com
-  events:
-    - type: "measurement_age"
-      target: "all"    # all sensors
-      threshold: "15m" # any duration
-    - type: "measurement_faulty"
-      target: "all"
-      threshold: ""
-
-# only needed when sending notifications via email
-email:
-  host: smtp.example.com
-  port: 25
-  user: foo
-  pass: bar
-  from: hildegunst@example.com
-```
-
-### possible values for `notifications`:
-`transport` | `options`
-------------|------------
-`email`     | `recipients`: list of email addresses
-
-Want more? [add it](#contribute)!
+Run `osem_notify help config` for details and an example configuration.
 
-### possible values for `events[]`:
+### available healthchecks
 
 `type`               | description
 ---------------------|------------
-`measurement_age`    | Alert when sensor `target` has not submitted measurements within `threshold` duration.
-`measurement_faulty` | Alert when sensor `target`'s last reading was a presumably faulty value (e.g. broken / disconnected sensor).
-`measurement_min`    | Alert when sensor `target`'s last measurement is lower than `threshold`.
-`measurement_max`    | Alert when sensor `target`'s last measurement is higher than `threshold`.
-
-`target` can be either a sensor ID, or `"all"` to match all sensors of the box.
-`threshold` must be a string.
+`measurement_age`    | Alert when a sensor has not submitted measurements within a given duration.
+`measurement_faulty` | Alert when a sensor's last reading was a presumably faulty value (e.g. broken / disconnected sensor).
+`measurement_min`    | Alert when a sensor's last measurement is lower than a given value.
+`measurement_max`    | Alert when a sensor's last measurement is higher than a given value.
 
-### configuration via environment variables
-Instead of a YAML file, you may configure the tool through environment variables. Keys are the same as in the YAML, but:
-
-- prefixed with `osem_notify_`
-- path separator is not `.`, but `_`
+### available notification transports
+`transport` | `options`
+------------|------------
+`email`     | `recipients`: list of email addresses
+`slack`     | -
+`xmpp`      | `recipients`: list of JIDs
 
-Example: `OSEM_NOTIFY_EMAIL_PASS=supersecret osem_notify check boxes`
+Want more? [add it](#contribute)!
 
 ## build it
 Want to use `osem_notify` on a platform where no builds are provided?
@@ -92,7 +63,7 @@ Contributions are welcome!
 Check out the following locations for plugging in new functionality:
 
 - new notification transports: [core/notifiers.go](core/notifiers.go)
-- new health checks: [core/Box.go](core/Box.go)
+- new health checks: [core/healthcheck*.go](core/healthchecks.go)
 - new commands: [cmd/](cmd/)
 
 Before committing and submitting a pull request, please run `go fmt ./ cmd/ core/`.

cmd/check.go

@@ -1,27 +0,0 @@
-package cmd
-
-import (
-	"github.com/spf13/cobra"
-)
-
-func init() {
-	checkCmd.AddCommand(checkBoxCmd)
-	rootCmd.AddCommand(checkCmd)
-}
-
-var checkCmd = &cobra.Command{
-	Use:   "check",
-	Short: "One-off check for events on boxes",
-	Long:  "One-off check for events on boxes",
-}
-
-var checkBoxCmd = &cobra.Command{
-	Use:   "boxes <boxId> [...<boxIds>]",
-	Short: "one-off check on one or more box IDs",
-	Long:  "specify box IDs to check them for events",
-	Args:  BoxIdValidator,
-	RunE: func(cmd *cobra.Command, args []string) error {
-		cmd.SilenceUsage = true
-		return checkAndNotify(args)
-	},
-}

cmd/cmd_check.go

@@ -0,0 +1,46 @@
+package cmd
+
+import (
+	"github.com/spf13/cobra"
+)
+
+func init() {
+	checkAllCmd.PersistentFlags().StringVarP(&date, "date", "", "", "filter boxes by date AND phenomenon")
+	checkAllCmd.PersistentFlags().StringVarP(&exposure, "exposure", "", "", "filter boxes by exposure")
+	checkAllCmd.PersistentFlags().StringVarP(&grouptag, "grouptag", "", "", "filter boxes by grouptag")
+	checkAllCmd.PersistentFlags().StringVarP(&model, "model", "", "", "filter boxes by model")
+	checkAllCmd.PersistentFlags().StringVarP(&phenomenon, "phenomenon", "", "", "filter boxes by phenomenon AND date")
+	checkCmd.AddCommand(checkBoxCmd)
+	checkCmd.AddCommand(checkAllCmd)
+	rootCmd.AddCommand(checkCmd)
+}
+
+var checkCmd = &cobra.Command{
+	Use:   "check",
+	Short: "One-off check for events on boxes",
+	Long:  "One-off check for events on boxes",
+}
+
+var checkBoxCmd = &cobra.Command{
+	Use:     "boxes <boxId> [...<boxIds>]",
+	Aliases: []string{"box"},
+	Short:   "one-off check on one or more box IDs",
+	Long:    "specify box IDs to check them for events",
+	Args:    BoxIdValidator,
+	RunE: func(cmd *cobra.Command, args []string) error {
+		cmd.SilenceUsage = true
+		return checkAndNotify(args)
+	},
+}
+
+var checkAllCmd = &cobra.Command{
+	Use:   "all",
+	Short: "one-off check on all boxes registered on the opensensemap instance",
+	RunE: func(cmd *cobra.Command, args []string) error {
+		cmd.SilenceUsage = true
+
+		// no flag validation, as the API already does a good job at that
+
+		return checkAndNotifyAll(parseBoxFilters())
+	},
+}

cmd/cmd_debug.go

@@ -0,0 +1,90 @@
+package cmd
+
+import (
+	"fmt"
+	"os"
+
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
+
+	"github.com/noerw/osem_notify/core"
+	"github.com/noerw/osem_notify/utils"
+)
+
+var (
+	clearCache bool
+)
+
+func init() {
+	debugCmd.AddCommand(debugNotificationsCmd)
+	debugCacheCmd.PersistentFlags().BoolVarP(&clearCache, "clear", "", false, "reset the notifications cache")
+	debugCmd.AddCommand(debugCacheCmd)
+	rootCmd.AddCommand(debugCmd)
+}
+
+var debugCmd = &cobra.Command{
+	Use:   "debug",
+	Short: "Run some debugging checks on osem_notify itself",
+	Long:  "osem_notify debug <feature> tests the functionality of the given feature",
+	PersistentPreRun: func(cmd *cobra.Command, args []string) {
+		log.SetLevel(log.DebugLevel)
+	},
+	PersistentPostRun: func(cmd *cobra.Command, args []string) {
+		utils.PrintConfig()
+	},
+	Run: func(cmd *cobra.Command, args []string) {
+		cmd.Help()
+	},
+}
+
+var debugCacheCmd = &cobra.Command{
+	Use:   "cache",
+	Short: "Print or clear the notifications cache",
+	Long:  "osem_notify debug cache prints the contents of the notifications cache",
+	RunE: func(cmd *cobra.Command, args []string) error {
+		if clearCache {
+			return core.ClearCache()
+		}
+		core.PrintCache()
+		return nil
+	},
+}
+
+var debugNotificationsCmd = &cobra.Command{
+	Use:   "notifications",
+	Short: "Verify that notifications are working",
+	Long: `osem_notify debug notifications sends a test notification according
+to healthchecks.default.notifications.options as defined in the config file`,
+	RunE: func(cmd *cobra.Command, args []string) error {
+		defaultNotifyConf := &core.NotifyConfig{}
+		err := viper.UnmarshalKey("healthchecks.default", defaultNotifyConf)
+		if err != nil {
+			return err
+		}
+
+		for transport, notifier := range core.Notifiers {
+			notLog := log.WithField("transport", transport)
+			opts := defaultNotifyConf.Notifications
+			notLog.Infof("testing notifer %s with options %v", transport, opts.Options)
+			n, err := notifier.New(opts)
+			if err != nil {
+				notLog.Warnf("could not initialize %s notifier: %s", transport, err)
+				continue
+			}
+
+			host, _ := os.Hostname()
+			err = n.Submit(core.Notification{
+				Subject: "Test notification from openSenseMap notifier",
+				Body:    fmt.Sprintf("Your notification set up on %s is working fine!", host),
+			})
+			if err != nil {
+				notLog.Warnf("could not submit test notification for %s notifier: %s", transport, err)
+				continue
+			}
+			notLog.Info("Test notification (successfully?) submitted, check the specified inbox")
+		}
+
+		return nil
+	},
+}

cmd/cmd_root.go

@@ -0,0 +1,173 @@
+package cmd
+
+import (
+	"os"
+
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/cobra"
+	// "github.com/spf13/cobra/doc"
+	"github.com/spf13/viper"
+
+	"github.com/noerw/osem_notify/utils"
+)
+
+var configHelpCmd = &cobra.Command{
+	Use:   "config",
+	Short: "How to configure osem_notify",
+	Long: `osem_notify works out of the box for basic functionality, but uses configuration to
+  set up notification transports and healthchecks. Additionally, all command line flags can
+  be set to default values through the configuration.
+
+  Configuration can be set either through a YAML file, or through environment variables.
+  You can use different configuration files per call by settings the --config flag.
+
+
+> Example configuration:
+
+  healthchecks:
+    # override default health checks for all boxes
+    default:
+      notifications:
+        transport: email
+        options:
+          recipients:
+          - fridolina@example.com
+      events:
+        - type: "measurement_age"
+          target: "all"    # all sensors
+          threshold: "15m" # any duration
+        - type: "measurement_faulty"
+          target: "all"
+          threshold: ""
+
+    # set health checks per box
+    593bcd656ccf3b0011791f5a:
+      notifications:
+        options:
+          recipients:
+          - ruth.less@example.com
+      events:
+        - type: "measurement_max"
+          target: "593bcd656ccf3b0011791f5b"
+          threshold: "40"
+
+  # only needed when sending notifications via email
+  email:
+    host: smtp.example.com
+    port: 25
+    user: foo
+    pass: bar
+    from: hildegunst@example.com
+
+  # only needed when sending notifications via Slack
+  slack:
+    webhook: https://hooks.slack.com/services/T030YPW07/xxxxxxx/xxxxxxxxxxxxxxxxxxxxxx
+
+  # only needed when sending notifications via XMPP
+  xmpp:
+    host: jabber.example.com:5222
+    user: foo@jabber.example.com
+    pass: bar
+    startls: true
+
+
+> possible values for healthchecks.*.notifications:
+
+  transport | options
+  ----------|-------------------------------------
+  email     | recipients: list of email addresses
+  slack     | -
+  xmpp      | recipients: list of JIDs
+
+
+> possible values for healthchecks.*.events[]:
+
+  type               | description
+  -------------------|---------------------------------------------------
+  measurement_age    | Alert when sensor target has not submitted measurements within threshold duration.
+  measurement_faulty | Alert when sensor target's last reading was a presumably faulty value (e.g. broken / disconnected sensor).
+  measurement_min    | Alert when sensor target's last measurement is lower than threshold.
+  measurement_max    | Alert when sensor target's last measurement is higher than threshold.
+
+  - target can be either a sensor ID, or "all" to match all sensors of the box.
+  - threshold must be a string.
+
+> configuration via environment variables
+
+  Instead of a YAML file, you may configure the tool through environment variables. Keys are the same as in the YAML, but:
+  keys are prefixed with "OSEM_NOTIFY_", path separator is not ".", but "_", all upper case
+
+  Example: OSEM_NOTIFY_EMAIL_PASS=supersecret osem_notify check boxes`,
+}
+
+var rootCmd = &cobra.Command{
+	Use:     "osem_notify",
+	Short:   "Root command displaying help",
+	Long:    "Run healthchecks and send notifications for boxes on opensensemap.org",
+	Version: "1.3.0",
+	PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
+		// set up logger
+		log.SetOutput(os.Stdout)
+		if viper.GetBool("debug") {
+			log.SetLevel(log.DebugLevel)
+			utils.PrintConfig()
+		} else {
+			log.SetLevel(log.InfoLevel)
+		}
+		switch viper.Get("logformat") {
+		case "json":
+			log.SetFormatter(&log.JSONFormatter{})
+		}
+
+		return nil
+	},
+	Run: func(cmd *cobra.Command, args []string) {
+		cmd.Help()
+	},
+}
+
+// accessed in initConfig(), as it is initialized before config is loaded (sic)
+var cfgFile string
+
+func init() {
+	var (
+		debug        bool
+		noCache      bool
+		shouldNotify string
+		logFormat    string
+		api          string
+	)
+
+	cobra.OnInitialize(initConfig)
+
+	rootCmd.PersistentFlags().StringVarP(&cfgFile, "config", "c", "", "path to config file (default $HOME/.osem_notify.yml)")
+	rootCmd.PersistentFlags().StringVarP(&api, "api", "a", "https://api.opensensemap.org", "openSenseMap API to query against")
+	rootCmd.PersistentFlags().StringVarP(&logFormat, "logformat", "l", "plain", "log format, can be plain or json")
+	rootCmd.PersistentFlags().BoolVarP(&debug, "debug", "d", false, "enable verbose logging")
+	rootCmd.PersistentFlags().StringVarP(&shouldNotify, "notify", "n", "", `If set, will send out notifications for the specified type of check result,
+otherwise results are printed to stdout only.
+Allowed values are "all", "error", "ok".
+You might want to run 'osem_notify debug notifications' first to verify everything works.
+
+Notifications for failing checks are sent only once, and then cached until the issue got
+resolved, unless --no-cache is set.
+To clear the cache, run 'osem_notify debug cache --clear'.
+`)
+	rootCmd.PersistentFlags().BoolVarP(&noCache, "no-cache", "", false, "send all notifications, ignoring results from previous runs. also don't update the cache.")
+
+	viper.BindPFlags(rootCmd.PersistentFlags()) // let flags override config
+
+	rootCmd.AddCommand(configHelpCmd)
+}
+
+func Execute() {
+	// generate documentation
+	// err := doc.GenMarkdownTree(rootCmd, "./docs")
+	// if err != nil {
+	// 	log.Fatal(err)
+	// }
+
+	if err := rootCmd.Execute(); err != nil {
+		os.Exit(1)
+	}
+}

cmd/cmd_watch.go

@@ -0,0 +1,87 @@
+package cmd
+
+import (
+	"time"
+
+	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
+)
+
+var ticker <-chan time.Time
+
+func init() {
+	var (
+		watchInterval int
+	)
+
+	watchAllCmd.PersistentFlags().StringVarP(&date, "date", "", "", "filter boxes by date")
+	watchAllCmd.PersistentFlags().StringVarP(&exposure, "exposure", "", "", "filter boxes by exposure")
+	watchAllCmd.PersistentFlags().StringVarP(&grouptag, "grouptag", "", "", "filter boxes by grouptag")
+	watchAllCmd.PersistentFlags().StringVarP(&model, "model", "", "", "filter boxes by model")
+	watchAllCmd.PersistentFlags().StringVarP(&phenomenon, "phenomenon", "", "", "filter boxes by phenomenon")
+	watchCmd.PersistentFlags().IntVarP(&watchInterval, "interval", "i", 30, "interval to run checks in minutes")
+	viper.BindPFlags(watchCmd.PersistentFlags())
+
+	watchCmd.AddCommand(watchBoxesCmd)
+	watchCmd.AddCommand(watchAllCmd)
+	rootCmd.AddCommand(watchCmd)
+}
+
+var watchCmd = &cobra.Command{
+	Use:     "watch",
+	Aliases: []string{"serve"},
+	Short:   "Watch boxes for events at an interval",
+	Long:    "Watch boxes for events at an interval",
+}
+
+var watchBoxesCmd = &cobra.Command{
+	Use:     "boxes <boxId> [...<boxIds>]",
+	Aliases: []string{"box"},
+	Short:   "watch a list of box IDs for events",
+	Long:    "specify box IDs to watch them for events",
+	Args:    BoxIdValidator,
+	PreRun: func(cmd *cobra.Command, args []string) {
+		interval := viper.GetDuration("interval") * time.Minute
+		ticker = time.NewTicker(interval).C
+	},
+	RunE: func(cmd *cobra.Command, args []string) error {
+		cmd.SilenceUsage = true
+		err := checkAndNotify(args)
+		if err != nil {
+			return err
+		}
+		for {
+			<-ticker
+			err = checkAndNotify(args)
+			if err != nil {
+				// we already did retries, so exiting seems appropriate
+				return err
+			}
+		}
+	},
+}
+
+var watchAllCmd = &cobra.Command{
+	Use:   "all",
+	Short: "watch all boxes registered on the map",
+	PreRun: func(cmd *cobra.Command, args []string) {
+		interval := viper.GetDuration("interval") * time.Minute
+		ticker = time.NewTicker(interval).C
+	},
+	RunE: func(cmd *cobra.Command, args []string) error {
+		cmd.SilenceUsage = true
+		filters := parseBoxFilters()
+		err := checkAndNotifyAll(filters)
+		if err != nil {
+			return err
+		}
+		for {
+			<-ticker
+			err = checkAndNotifyAll(filters)
+			if err != nil {
+				// we already did retries, so exiting seems appropriate
+				return err
+			}
+		}
+	},
+}

cmd/config.go

@@ -2,69 +2,20 @@ package cmd
 
 import (
 	"os"
-	"path"
 	"strings"
 
 	log "github.com/sirupsen/logrus"
 	"github.com/spf13/viper"
-)
-
-/**
- * config file handling, as it is kinda broken in spf13/viper
- * mostly copied from https://github.com/TheThingsNetwork/ttn/blob/f623a6a/ttnctl/util/config.go
- */
-
-// GetConfigFile returns the location of the configuration file.
-// It checks the following (in this order):
-// the --config flag
-// $XDG_CONFIG_HOME/osem_notify/config.yml (if $XDG_CONFIG_HOME is set)
-// $HOME/.osem_notify.yml
-func getConfigFile() string {
-	flag := viper.GetString("config")
-
-	xdg := os.Getenv("XDG_CONFIG_HOME")
-	if xdg != "" {
-		xdg = path.Join(xdg, "osem_notify", "config.yml")
-	}
-
-	home := os.Getenv("HOME")
-	homeyml := ""
-	homeyaml := ""
-
-	if home != "" {
-		homeyml = path.Join(home, ".osem_notify.yml")
-		homeyaml = path.Join(home, ".osem_notify.yaml")
-	}
-
-	try_files := []string{
-		flag,
-		xdg,
-		homeyml,
-		homeyaml,
-	}
 
-	// find a file that exists, and use that
-	for _, file := range try_files {
-		if file != "" {
-			if _, err := os.Stat(file); err == nil {
-				return file
-			}
-		}
-	}
-
-	// no file found, set up correct fallback
-	if os.Getenv("XDG_CONFIG_HOME") != "" {
-		return xdg
-	} else {
-		return homeyml
-	}
-}
+	"github.com/noerw/osem_notify/core"
+	"github.com/noerw/osem_notify/utils"
+)
 
 // initConfig reads in config file and ENV variables if set.
 func initConfig() {
 	theConfig := cfgFile
 	if cfgFile == "" {
-		theConfig = getConfigFile()
+		theConfig = utils.GetConfigFile("osem_notify")
 	}
 
 	viper.SetConfigType("yaml")
@@ -88,36 +39,72 @@ func initConfig() {
 }
 
 func validateConfig() {
-	transport := viper.GetString("defaultHealthchecks.notifications.transport")
-	if viper.GetBool("notify") && transport == "email" {
-		if len(viper.GetStringSlice("defaultHealthchecks.notifications.options.recipients")) == 0 {
-			log.Warn("No recipients set up for transport email")
+	if viper.GetString("notify") != "" {
+		if len(viper.GetStringSlice("healthchecks.default.notifications.options.recipients")) == 0 {
+			log.Warn("No default recipients set up for notifications!")
 		}
 
-		emailRequired := []string{
-			viper.GetString("email.host"),
-			viper.GetString("email.port"),
-			viper.GetString("email.user"),
-			viper.GetString("email.pass"),
-			viper.GetString("email.from"),
+		var conf = &core.TransportConfig{}
+		if err := viper.UnmarshalKey("healthchecks.default.notifications", conf); err != nil {
+			log.Error("invalid default notification configuration: ", err)
+			os.Exit(1)
 		}
-		for _, conf := range emailRequired {
-			if conf == "" {
-				log.Error("Default transport set as email, but missing email config")
-				os.Exit(1)
-			}
+
+		// creating a notifier validates its configuration
+		_, err := core.GetNotifier(conf)
+		if err != nil {
+			log.Error(err)
+			os.Exit(1)
 		}
 	}
 }
 
-func printConfig() {
-	log.Debug("Using config:")
-	printKV("config file", viper.ConfigFileUsed())
-	for key, val := range viper.AllSettings() {
-		printKV(key, val)
+func getNotifyConf(boxID string) (*core.NotifyConfig, error) {
+	// config used when no configuration is present at all
+	conf := &core.NotifyConfig{
+		Events: []core.NotifyEvent{
+			core.NotifyEvent{
+				Type:      "measurement_age",
+				Target:    "all",
+				Threshold: "15m",
+			},
+			core.NotifyEvent{
+				Type:      "measurement_faulty",
+				Target:    "all",
+				Threshold: "",
+			},
+		},
+	}
+
+	// override with default configuration from file
+	// considering the case that .events may be defined but empty
+	// to allow to define no events, and don't leak shorter lists into
+	// previous longer ones
+	if keyDefined("healthchecks.default.events") {
+		conf.Events = []core.NotifyEvent{}
+	}
+	if err := viper.UnmarshalKey("healthchecks.default", conf); err != nil {
+		return nil, err
 	}
+
+	// override with per box configuration from file
+	if keyDefined("healthchecks." + boxID + ".events") {
+		conf.Events = []core.NotifyEvent{}
+	}
+	if err := viper.UnmarshalKey("healthchecks."+boxID, conf); err != nil {
+		return nil, err
+	}
+
+	return conf, nil
 }
 
-func printKV(key, val interface{}) {
-	log.Debugf("%20s: %v", key, val)
+// implement our own keyCheck, as viper.InConfig() does not work
+func keyDefined(key string) bool {
+	allConfKeys := viper.AllKeys()
+	for _, k := range allConfKeys {
+		if k == key {
+			return true
+		}
+	}
+	return false
 }

cmd/root.go

@@ -1,66 +0,0 @@
-package cmd
-
-import (
-	"os"
-
-	log "github.com/sirupsen/logrus"
-	"github.com/spf13/cobra"
-	"github.com/spf13/viper"
-)
-
-var rootCmd = &cobra.Command{
-	Use:   "osem_notify",
-	Short: "Root command displaying help",
-	Long:  "Run healthchecks and send notifications for boxes on opensensemap.org",
-	PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
-		// set up logger
-		log.SetOutput(os.Stdout)
-		if viper.GetBool("debug") {
-			log.SetLevel(log.DebugLevel)
-			printConfig()
-		} else {
-			log.SetLevel(log.InfoLevel)
-		}
-		switch viper.Get("logformat") {
-		case "json":
-			log.SetFormatter(&log.JSONFormatter{})
-		}
-
-		return nil
-	},
-	Run: func(cmd *cobra.Command, args []string) {
-		cmd.Help()
-	},
-}
-
-// accessed in initConfig(), as it is initialized before config is loaded (sic)
-var cfgFile string
-
-func init() {
-	var (
-		shouldNotify bool
-		debug        bool
-		logFormat    string
-	)
-
-	cobra.OnInitialize(initConfig)
-
-	rootCmd.PersistentFlags().StringVarP(&cfgFile, "config", "c", "", "path to config file (default $HOME/.osem_notify.yml)")
-	rootCmd.PersistentFlags().StringVarP(&logFormat, "logformat", "l", "plain", "log format, can be plain or json")
-	rootCmd.PersistentFlags().BoolVarP(&debug, "debug", "d", false, "enable verbose logging")
-	rootCmd.PersistentFlags().BoolVarP(&shouldNotify, "notify", "n", false, "if set, will send out notifications.\nOtherwise results are printed to stdout only")
-
-	viper.BindPFlags(rootCmd.PersistentFlags()) // let flags override config
-}
-
-func Execute() {
-	// generate documentation
-	// err := doc.GenMarkdownTree(rootCmd, "./doc")
-	// if err != nil {
-	// 	log.Fatal(err)
-	// }
-
-	if err := rootCmd.Execute(); err != nil {
-		os.Exit(1)
-	}
-}

cmd/shared.go

@@ -3,7 +3,9 @@ package cmd
 import (
 	"fmt"
 	"regexp"
+	"strings"
 
+	log "github.com/sirupsen/logrus"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
 
@@ -32,22 +34,89 @@ func BoxIdValidator(cmd *cobra.Command, args []string) error {
 	return nil
 }
 
-func checkAndNotify(boxIds []string) error {
-	defaultNotifyConf := &core.NotifyConfig{}
-	err := viper.UnmarshalKey("defaultHealthchecks", defaultNotifyConf)
+func checkAndNotifyAll(filters core.BoxFilters) error {
+	log.Info("getting list of boxes...")
+
+	// fetch all boxes first & extract their IDs
+	osem := core.NewOsemClient(viper.GetString("api"))
+	boxes, err := osem.GetAllBoxes(filters)
 	if err != nil {
 		return err
 	}
+	boxIDs := make([]string, len(*boxes))
+	for i, box := range *boxes {
+		boxIDs[i] = box.Id
+	}
 
-	results, err := core.CheckBoxes(boxIds, defaultNotifyConf)
+	// then check each box individually. we only pass the ID
+	// and fetch again, because box metadata is different in
+	// GetAllBoxes and GetBox..
+	return checkAndNotify(boxIDs)
+}
+
+func checkAndNotify(boxIds []string) error {
+	boxLocalConfig := map[string]*core.NotifyConfig{}
+	for _, boxID := range boxIds {
+		c, err := getNotifyConf(boxID)
+		if err != nil {
+			return err
+		}
+		boxLocalConfig[boxID] = c
+	}
+
+	osem := core.NewOsemClient(viper.GetString("api"))
+	results, err := core.CheckBoxes(boxLocalConfig, osem)
 	if err != nil {
 		return err
 	}
 
 	results.Log()
 
-	if viper.GetBool("notify") {
-		return results.SendNotifications()
+	notify := strings.ToLower(viper.GetString("notify"))
+	if notify != "" {
+		types := []string{}
+		switch notify {
+		case "all":
+			types = []string{core.CheckErr, core.CheckOk}
+		case "error", "err":
+			types = []string{core.CheckErr}
+		case "ok":
+			types = []string{core.CheckOk}
+		default:
+			return fmt.Errorf("invalid value %s for \"notify\"", notify)
+		}
+
+		useCache := !viper.GetBool("no-cache")
+		return results.SendNotifications(types, useCache)
 	}
 	return nil
 }
+
+var ( // values are set during cli flag parsing of checkAllCmd & watchAllCmd
+	date       string
+	exposure   string
+	grouptag   string
+	model      string
+	phenomenon string
+)
+
+func parseBoxFilters() core.BoxFilters {
+	filters := core.BoxFilters{}
+	if date != "" {
+		filters.Date = date // TODO: parse date & format as ISO date?
+	}
+	if exposure != "" {
+		filters.Exposure = exposure
+	}
+	if grouptag != "" {
+		filters.Grouptag = grouptag
+	}
+	if model != "" {
+		filters.Model = model
+	}
+	if phenomenon != "" {
+		filters.Phenomenon = phenomenon
+	}
+
+	return filters
+}

cmd/watch.go

@@ -1,47 +0,0 @@
-package cmd
-
-import (
-	"time"
-
-	"github.com/spf13/cobra"
-)
-
-var watchInterval int
-var ticker <-chan time.Time
-
-func init() {
-	watchCmd.AddCommand(watchBoxesCmd)
-	watchCmd.PersistentFlags().IntVarP(&watchInterval, "interval", "i", 15, "interval to run checks in minutes")
-	rootCmd.AddCommand(watchCmd)
-}
-
-var watchCmd = &cobra.Command{
-	Use:     "watch",
-	Aliases: []string{"serve"},
-	Short:   "Watch boxes for events at an interval",
-	Long:    "Watch boxes for events at an interval",
-}
-
-var watchBoxesCmd = &cobra.Command{
-	Use:   "boxes <boxId> [...<boxIds>]",
-	Short: "watch a list of box IDs for events",
-	Long:  "specify box IDs to watch them for events",
-	Args:  BoxIdValidator,
-	PreRun: func(cmd *cobra.Command, args []string) {
-		ticker = time.NewTicker(time.Duration(watchInterval) * time.Second).C
-	},
-	RunE: func(cmd *cobra.Command, args []string) error {
-		cmd.SilenceUsage = true
-		err := checkAndNotify(args)
-		if err != nil {
-			return err
-		}
-		for {
-			<-ticker
-			err = checkAndNotify(args)
-			if err != nil {
-				return err
-			}
-		}
-	},
-}

core/Box.go

@@ -1,161 +0,0 @@
-package core
-
-import (
-	"fmt"
-	"strconv"
-	"time"
-)
-
-const (
-	CheckOk                   = "OK"
-	CheckErr                  = "FAILED"
-	eventMeasurementAge       = "measurement_age"
-	eventMeasurementValMin    = "measurement_min"
-	eventMeasurementValMax    = "measurement_max"
-	eventMeasurementValFaulty = "measurement_faulty"
-	eventTargetAll            = "all" // if event.Target is this value, all sensors will be checked
-)
-
-type checkType = struct{ description string }
-
-var checkTypes = map[string]checkType{
-	eventMeasurementAge:       checkType{"No measurement from %s (%s) since %s"},
-	eventMeasurementValMin:    checkType{"Sensor %s (%s) reads low value of %s"},
-	eventMeasurementValMax:    checkType{"Sensor %s (%s) reads high value of %s"},
-	eventMeasurementValFaulty: checkType{"Sensor %s (%s) reads presumably faulty value of %s"},
-}
-
-type FaultyValue struct {
-	sensor string
-	val    float64
-}
-
-var faultyVals = map[FaultyValue]bool{
-	FaultyValue{sensor: "BMP280", val: 0.0}:  true,
-	FaultyValue{sensor: "HDC1008", val: 0.0}: true,
-	FaultyValue{sensor: "HDC1008", val: -40}: true,
-	FaultyValue{sensor: "SDS 011", val: 0.0}: true,
-}
-
-type NotifyEvent struct {
-	Type      string `json:"type"`
-	Target    string `json:"target"`
-	Threshold string `json:"threshold"`
-}
-
-type TransportConfig struct {
-	Transport string      `json:"transport"`
-	Options   interface{} `json:"options"`
-}
-
-type NotifyConfig struct {
-	Notifications TransportConfig `json:"notifications"`
-	Events        []NotifyEvent   `json:"events"`
-}
-
-type Box struct {
-	Id      string `json:"_id"`
-	Name    string `json:"name"`
-	Sensors []struct {
-		Id              string `json:"_id"`
-		Phenomenon      string `json:"title"`
-		Type            string `json:"sensorType"`
-		LastMeasurement *struct {
-			Value string    `json:"value"`
-			Date  time.Time `json:"createdAt"`
-		} `json:"lastMeasurement"`
-	} `json:"sensors"`
-	NotifyConf *NotifyConfig `json:"healthcheck"`
-}
-
-func (box Box) RunChecks() ([]CheckResult, error) {
-	var results = []CheckResult{}
-
-	for _, event := range box.NotifyConf.Events {
-		for _, s := range box.Sensors {
-			// if a sensor never measured anything, thats ok. checks would fail anyway
-			if s.LastMeasurement == nil {
-				continue
-			}
-
-			// a validator must set these values
-			var (
-				status     = CheckOk
-				target     = s.Id
-				targetName = s.Phenomenon
-				value      string
-			)
-
-			if event.Target == eventTargetAll || event.Target == s.Id {
-
-				switch event.Type {
-				case eventMeasurementAge:
-					thresh, err := time.ParseDuration(event.Threshold)
-					if err != nil {
-						return nil, err
-					}
-					if time.Since(s.LastMeasurement.Date) > thresh {
-						status = CheckErr
-					}
-
-					value = s.LastMeasurement.Date.String()
-
-				case eventMeasurementValMin, eventMeasurementValMax:
-					thresh, err := strconv.ParseFloat(event.Threshold, 64)
-					if err != nil {
-						return nil, err
-					}
-					val, err2 := strconv.ParseFloat(s.LastMeasurement.Value, 64)
-					if err2 != nil {
-						return nil, err2
-					}
-					if event.Type == eventMeasurementValMax && val > thresh ||
-						event.Type == eventMeasurementValMin && val < thresh {
-						status = CheckErr
-					}
-
-					value = s.LastMeasurement.Value
-
-				case eventMeasurementValFaulty:
-					val, err := strconv.ParseFloat(s.LastMeasurement.Value, 64)
-					if err != nil {
-						return nil, err
-					}
-					if faultyVals[FaultyValue{
-						sensor: s.Type,
-						val:    val,
-					}] {
-						status = CheckErr
-					}
-
-					value = s.LastMeasurement.Value
-				}
-
-				results = append(results, CheckResult{
-					Threshold:  event.Threshold,
-					Event:      event.Type,
-					Target:     target,
-					TargetName: targetName,
-					Value:      value,
-					Status:     status,
-				})
-			}
-		}
-	}
-
-	return results, nil
-}
-
-func (box Box) GetNotifier() (AbstractNotifier, error) {
-	transport := box.NotifyConf.Notifications.Transport
-	if transport == "" {
-		return nil, fmt.Errorf("No notification transport provided")
-	}
-
-	notifier := notifiers[transport]
-	if notifier == nil {
-		return nil, fmt.Errorf("%s is not a supported notification transport", transport)
-	}
-
-	return notifier.New(box.NotifyConf.Notifications.Options)
-}

core/BoxCheckResults.go

@@ -1,184 +0,0 @@
-package core
-
-import (
-	"crypto/sha256"
-	"encoding/hex"
-	"fmt"
-
-	log "github.com/sirupsen/logrus"
-	"github.com/spf13/viper"
-)
-
-type CheckResult struct {
-	Status     string
-	Event      string
-	Target     string
-	TargetName string
-	Value      string
-	Threshold  string
-}
-
-func (r CheckResult) EventID() string {
-	s := fmt.Sprintf("%s%s%s", r.Event, r.Target, r.Threshold)
-	hasher := sha256.New()
-	hasher.Write([]byte(s))
-	return hex.EncodeToString(hasher.Sum(nil))
-}
-
-func (r CheckResult) String() string {
-	if r.Status == CheckOk {
-		return fmt.Sprintf("%s %s (on sensor %s (%s) with value %s)\n", r.Event, r.Status, r.TargetName, r.Target, r.Value)
-	} else {
-		return fmt.Sprintf("%s: "+checkTypes[r.Event].description+"\n", r.Status, r.TargetName, r.Target, r.Value)
-	}
-}
-
-type BoxCheckResults map[*Box][]CheckResult
-
-func (results BoxCheckResults) Size() int {
-	size := 0
-	for _, boxResults := range results {
-		size += len(boxResults)
-	}
-	return size
-}
-
-func (results BoxCheckResults) Log() {
-	for box, boxResults := range results {
-		boxLog := log.WithFields(log.Fields{
-			"boxId": box.Id,
-		})
-		countErr := 0
-		for _, r := range boxResults {
-			resultLog := boxLog.WithFields(log.Fields{
-				"status": r.Status,
-				"event":  r.Event,
-				"value":  r.Value,
-				"target": r.Target,
-			})
-			if r.Status == CheckOk {
-				resultLog.Debugf("%s: %s", box.Name, r)
-			} else {
-				resultLog.Warnf("%s: %s", box.Name, r)
-				countErr++
-			}
-		}
-		if countErr == 0 {
-			boxLog.Infof("%s: all is fine!", box.Name)
-		}
-	}
-}
-
-func (results BoxCheckResults) SendNotifications() error {
-	// FIXME: don't return on errors, process all boxes first!
-	results = results.FilterChangedFromCache(false)
-
-	n := results.Size()
-	if n == 0 {
-		log.Info("No notifications due.")
-		return nil
-	} else {
-		log.Infof("Notifying for %v checks turned bad in total...", results.Size())
-	}
-
-	for box, resultsDue := range results {
-		if len(resultsDue) == 0 {
-			continue
-		}
-
-		transport := box.NotifyConf.Notifications.Transport
-		notifyLog := log.WithFields(log.Fields{
-			"boxId":     box.Id,
-			"transport": transport,
-		})
-
-		notifier, err2 := box.GetNotifier()
-		if err2 != nil {
-			notifyLog.Error(err2)
-			return err2
-		}
-		notification := notifier.ComposeNotification(box, resultsDue)
-		err3 := notifier.Submit(notification)
-		if err3 != nil {
-			notifyLog.Error(err3)
-			return err3
-		}
-		notifyLog.Infof("Sent notification for %s via %s with %v new issues", box.Name, transport, len(resultsDue))
-	}
-
-	return nil
-}
-
-func (results BoxCheckResults) FilterChangedFromCache(keepOk bool) BoxCheckResults {
-	remaining := BoxCheckResults{}
-
-	for box, boxResults := range results {
-		// get results from cache. they are indexed by an event ID per boxId
-		// filter, so that only changed result.Status remain
-		remaining[box] = []CheckResult{}
-		for _, result := range boxResults {
-			cached := viper.GetStringMap(fmt.Sprintf("watchcache.%s.%s", box.Id, result.EventID()))
-			if result.Status != cached["laststatus"] {
-				if result.Status != CheckOk || keepOk {
-					remaining[box] = append(remaining[box], result)
-				}
-			}
-		}
-
-		// TODO: reminder functionality: extract additional results with Status ERR
-		// from cache with time.Since(lastNotifyDate) > remindAfter.
-		// would require to serialize the full result..
-	}
-
-	// upate cache, setting lastNotifyDate to Now()
-	for box, boxResults := range results {
-		for _, result := range boxResults {
-			// FIXME: somehow this is not persisted?
-			key := fmt.Sprintf("watchcache.%s.%s", box.Id, result.EventID())
-			viper.Set(key+".laststatus", result.Status)
-		}
-	}
-
-	return remaining
-}
-
-func CheckBoxes(boxIds []string, defaultConf *NotifyConfig) (BoxCheckResults, error) {
-	log.Debug("Checking notifications for ", len(boxIds), " box(es)")
-
-	results := BoxCheckResults{}
-	for _, boxId := range boxIds {
-		box, res, err := checkBox(boxId, defaultConf)
-		if err != nil {
-			return nil, err
-		}
-		results[box] = res
-	}
-
-	return results, nil
-}
-
-func checkBox(boxId string, defaultConf *NotifyConfig) (*Box, []CheckResult, error) {
-	boxLogger := log.WithFields(log.Fields{"boxId": boxId})
-	boxLogger.Info("checking box for events")
-
-	// get box data
-	box, err := Osem.GetBox(boxId)
-	if err != nil {
-		boxLogger.Error(err)
-		return nil, nil, err
-	}
-
-	// if box has no notify config, we use the defaultConf
-	if box.NotifyConf == nil {
-		box.NotifyConf = defaultConf
-	}
-
-	// run checks
-	results, err2 := box.RunChecks()
-	if err2 != nil {
-		boxLogger.Error("could not run checks on box: ", err2)
-		return box, results, err2
-	}
-
-	return box, results, nil
-}

core/OsemClient.go

@@ -1,35 +0,0 @@
-package core
-
-import (
-	"errors"
-	"net/http"
-
-	"github.com/dghubble/sling"
-)
-
-type OsemError struct {
-	Code    int    `json:"code"`
-	Message string `json:"message"`
-}
-
-type OsemClient struct {
-	sling *sling.Sling
-}
-
-func NewOsemClient(client *http.Client) *OsemClient {
-	return &OsemClient{
-		sling: sling.New().Client(client).Base("https://api.opensensemap.org/"),
-	}
-}
-
-func (client *OsemClient) GetBox(boxId string) (*Box, error) {
-	box := &Box{}
-	fail := &OsemError{}
-	client.sling.New().Path("boxes/").Path(boxId).Receive(box, fail)
-	if fail.Message != "" {
-		return box, errors.New("could not fetch box: " + fail.Message)
-	}
-	return box, nil
-}
-
-var Osem = NewOsemClient(&http.Client{}) // default client

core/cache.go

@@ -0,0 +1,80 @@
+package core
+
+import (
+	"fmt"
+	"os"
+
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/viper"
+
+	"github.com/noerw/osem_notify/utils"
+)
+
+/**
+ * in memory + yaml persisted cache for check results, ensuring we don't resend
+ * notifications on every check
+
+ * TODO: reminder functionality: extract additional results with Status ERR
+ * from cache with time.Since(lastNotifyDate) > remindAfter.
+ * would require to serialize the full result..
+ */
+
+var cache = viper.New()
+
+func init() {
+	fileName := utils.GetConfigFile("osem_notify_cache")
+
+	cache.SetConfigType("yaml")
+	cache.SetConfigFile(fileName)
+
+	if _, err := os.Stat(fileName); err == nil {
+		err := cache.ReadInConfig()
+		if err != nil {
+			log.Error("Error when reading cache file:", err)
+		}
+	}
+}
+
+func (results BoxCheckResults) filterChangedFromCache() BoxCheckResults {
+	remaining := BoxCheckResults{}
+
+	for box, boxResults := range results {
+		// get results from cache. they are indexed by an event ID per boxId
+		// filter, so that only changed result.Status remain
+		remaining[box] = []CheckResult{}
+		for _, result := range boxResults {
+			cached := cache.GetStringMap(fmt.Sprintf("watchcache.%s.%s", box.Id, result.EventID()))
+			if result.Status != cached["laststatus"] {
+				remaining[box] = append(remaining[box], result)
+			}
+		}
+	}
+
+	return remaining
+}
+
+func updateCache(box *Box, results []CheckResult) {
+	for _, result := range results {
+		key := fmt.Sprintf("watchcache.%s.%s", box.Id, result.EventID())
+		cache.Set(key+".laststatus", result.Status)
+	}
+}
+
+func writeCache() error {
+	return cache.WriteConfig()
+}
+
+func ClearCache() error {
+	fileName := utils.GetConfigFile("osem_notify_cache")
+	_, err := os.Stat(fileName)
+	if err != nil {
+		return nil
+	}
+	return os.Remove(fileName)
+}
+
+func PrintCache() {
+	for key, val := range cache.AllSettings() {
+		log.Infof("%20s: %v", key, val)
+	}
+}

core/checkrunner.go

@@ -0,0 +1,133 @@
+package core
+
+import (
+	"fmt"
+	"strings"
+
+	log "github.com/sirupsen/logrus"
+)
+
+type BoxCheckResults map[*Box][]CheckResult
+
+func (results BoxCheckResults) Size(statusToCheck []string) int {
+	size := 0
+	for _, boxResults := range results {
+		for _, result := range boxResults {
+			if result.HasStatus(statusToCheck) {
+				size++
+			}
+		}
+	}
+	return size
+}
+
+func (results BoxCheckResults) Log() {
+	// collect statistics for summary print
+	boxesSkipped := 0
+	boxesWithIssues := 0
+	boxesWithoutIssues := 0
+	failedChecks := 0
+	errorsByEvent := map[string]int{}
+	for event, _ := range checkers {
+		errorsByEvent[event] = 0
+	}
+
+	for box, boxResults := range results {
+		boxLog := log.WithFields(log.Fields{
+			"boxId": box.Id,
+		})
+		countErr := 0
+		for _, r := range boxResults {
+			resultLog := boxLog.WithFields(log.Fields{
+				"status": r.Status,
+				"event":  r.Event,
+				"value":  r.Value,
+				"target": r.Target,
+			})
+			if r.Status == CheckOk {
+				resultLog.Debugf("%s: %s", box.Name, r)
+			} else {
+				resultLog.Warnf("%s: %s", box.Name, r)
+				countErr++
+				errorsByEvent[r.Event]++
+			}
+		}
+
+		if len(boxResults) == 0 {
+			boxLog.Infof("%s: no checks defined", box.Name)
+			boxesSkipped++
+		} else if countErr == 0 {
+			boxLog.Infof("%s: all is fine!", box.Name)
+			boxesWithoutIssues++
+		} else {
+			// we logged the error(s) already
+			boxesWithIssues++
+			failedChecks += countErr
+		}
+	}
+
+	// print summary
+	boxesChecked := boxesWithIssues + boxesWithoutIssues
+	if boxesChecked > 1 {
+		summaryLog := log.WithFields(log.Fields{
+			"boxesChecked":  boxesChecked,
+			"boxesSkipped":  boxesSkipped, // boxes are also skipped when they never submitted any measurements before!
+			"boxesOk":       boxesWithoutIssues,
+			"boxesErr":      boxesWithIssues,
+			"failedChecks":  failedChecks,
+			"errorsByEvent": errorsByEvent,
+		})
+		summaryLog.Infof(
+			"check summary: %v of %v checked boxes are fine (%v had no checks)!",
+			boxesWithoutIssues,
+			boxesChecked,
+			boxesSkipped)
+	}
+}
+
+func CheckBoxes(boxLocalConfs map[string]*NotifyConfig, osem *OsemClient) (BoxCheckResults, error) {
+	log.Info("Checking notifications for ", len(boxLocalConfs), " box(es)")
+
+	results := BoxCheckResults{}
+	errs := []string{}
+
+	// @TODO: check boxes in parallel, capped at 5 at once. and/or rate limit?
+	for boxId, localConf := range boxLocalConfs {
+		boxLogger := log.WithField("boxId", boxId)
+		boxLogger.Debug("checking box for events")
+
+		box, res, err := checkBox(boxId, localConf, osem)
+		if err != nil {
+			boxLogger.Errorf("could not run checks on box %s: %s", boxId, err)
+			errs = append(errs, err.Error())
+			continue
+		}
+		results[box] = res
+	}
+
+	if len(errs) != 0 {
+		return results, fmt.Errorf(strings.Join(errs, "\n"))
+	}
+	return results, nil
+}
+
+func checkBox(boxId string, defaultConf *NotifyConfig, osem *OsemClient) (*Box, []CheckResult, error) {
+	// get box data
+	box, err := osem.GetBox(boxId)
+	if err != nil {
+		return nil, nil, err
+	}
+
+	// if box has no notify config, we use the defaultConf
+	if box.NotifyConf == nil {
+		box.NotifyConf = defaultConf
+	}
+
+	// run checks
+	results, err2 := box.RunChecks()
+	if err2 != nil {
+		return box, results, err2
+	}
+
+	return box, results, nil
+}

core/healthcheck_measurement_age.go

@@ -0,0 +1,36 @@
+package core
+
+import (
+	"fmt"
+	"time"
+)
+
+var checkMeasurementAge = checkType{
+	name: "measurement_age",
+	toString: func(r CheckResult) string {
+		return fmt.Sprintf("No measurement from %s (%s) since %s", r.TargetName, r.Target, r.Value)
+	},
+	checkFunc: func(e NotifyEvent, s Sensor, b Box) (CheckResult, error) {
+		result := CheckResult{
+			Event:      e.Type,
+			Target:     s.Id,
+			TargetName: s.Phenomenon,
+			Threshold:  e.Threshold,
+			Value:      s.LastMeasurement.Date.String(),
+			Status:     CheckOk,
+		}
+
+		thresh, err := time.ParseDuration(e.Threshold)
+		if err != nil {
+			return CheckResult{}, err
+		}
+
+		if time.Since(s.LastMeasurement.Date) > thresh {
+			result.Status = CheckErr
+		}
+
+		result.Value = s.LastMeasurement.Date.String()
+
+		return result, nil
+	},
+}

core/healthcheck_measurement_faulty.go

@@ -0,0 +1,52 @@
+package core
+
+import (
+	"fmt"
+
+	"github.com/noerw/osem_notify/utils"
+)
+
+var checkMeasurementFaulty = checkType{
+	name: "measurement_faulty",
+	toString: func(r CheckResult) string {
+		return fmt.Sprintf("Sensor %s (%s) reads presumably faulty value of %s", r.TargetName, r.Target, r.Value)
+	},
+	checkFunc: func(e NotifyEvent, s Sensor, b Box) (CheckResult, error) {
+		result := CheckResult{
+			Event:      e.Type,
+			Target:     s.Id,
+			TargetName: s.Phenomenon,
+			Threshold:  e.Threshold,
+			Value:      s.LastMeasurement.Value,
+			Status:     CheckOk,
+		}
+
+		val, err := utils.ParseFloat(s.LastMeasurement.Value)
+		if err != nil {
+			return result, err
+		}
+
+		if faultyVals[faultyValue{
+			sensor: s.Type,
+			val:    val,
+		}] {
+			result.Status = CheckErr
+		}
+
+		return result, nil
+	},
+}
+
+type faultyValue struct {
+	sensor string
+	val    float64
+}
+
+var faultyVals = map[faultyValue]bool{
+	// @TODO: add UV & light sensor: check for 0 if not sunset based on boxlocation
+	// @TODO: add BME280 and other sensors..
+	faultyValue{sensor: "BMP280", val: 0.0}:  true,
+	faultyValue{sensor: "HDC1008", val: 0.0}: true, // @FIXME: check should be on luftfeuchte only!
+	faultyValue{sensor: "HDC1008", val: -40}: true,
+	faultyValue{sensor: "SDS 011", val: 0.0}: true, // @FIXME: 0.0 seems to be a correct value, need to check over longer periods
+}

core/healthcheck_measurement_minmax.go

@@ -0,0 +1,56 @@
+package core
+
+import (
+	"fmt"
+
+	"github.com/noerw/osem_notify/utils"
+)
+
+const (
+	nameMin = "measurement_min"
+	nameMax = "measurement_max"
+)
+
+var checkMeasurementMin = checkType{
+	name: nameMin,
+	toString: func(r CheckResult) string {
+		return fmt.Sprintf("Sensor %s (%s) reads low value of %s", r.TargetName, r.Target, r.Value)
+	},
+	checkFunc: validateMeasurementMinMax,
+}
+
+var checkMeasurementMax = checkType{
+	name: nameMax,
+	toString: func(r CheckResult) string {
+		return fmt.Sprintf("Sensor %s (%s) reads high value of %s", r.TargetName, r.Target, r.Value)
+	},
+	checkFunc: validateMeasurementMinMax,
+}
+
+func validateMeasurementMinMax(e NotifyEvent, s Sensor, b Box) (CheckResult, error) {
+	result := CheckResult{
+		Event:      e.Type,
+		Target:     s.Id,
+		TargetName: s.Phenomenon,
+		Threshold:  e.Threshold,
+		Value:      s.LastMeasurement.Value,
+		Status:     CheckOk,
+	}
+
+	thresh, err := utils.ParseFloat(e.Threshold)
+	if err != nil {
+		return result, err
+	}
+
+	val, err := utils.ParseFloat(s.LastMeasurement.Value)
+	if err != nil {
+		return result, err
+	}
+
+	if e.Type == nameMax && val > thresh ||
+		e.Type == nameMin && val < thresh {
+		result.Status = CheckErr
+	}
+
+	return result, nil
+}

core/healthchecks.go

@@ -0,0 +1,95 @@
+package core
+
+import (
+	"crypto/sha256"
+	"encoding/hex"
+	"fmt"
+
+	log "github.com/sirupsen/logrus"
+)
+
+const (
+	CheckOk        = "OK"
+	CheckErr       = "FAILED"
+	eventTargetAll = "all" // if event.Target is this value, all sensors will be checked
+)
+
+type checkType struct {
+	name      string                          // name that is used in config
+	toString  func(result CheckResult) string // error message when check failed
+	checkFunc func(event NotifyEvent, sensor Sensor, context Box) (CheckResult, error)
+}
+
+var checkers = map[string]checkType{
+	checkMeasurementAge.name:    checkMeasurementAge,
+	checkMeasurementMin.name:    checkMeasurementMin,
+	checkMeasurementMax.name:    checkMeasurementMax,
+	checkMeasurementFaulty.name: checkMeasurementFaulty,
+}
+
+type CheckResult struct {
+	Status     string // should be CheckOk | CheckErr
+	TargetName string
+	Value      string
+	Target     string
+
+	Event     string // these should be copied from the NotifyEvent
+	Threshold string
+}
+
+func (r CheckResult) HasStatus(statusToCheck []string) bool {
+	for _, status := range statusToCheck {
+		if status == r.Status {
+			return true
+		}
+	}
+	return false
+}
+
+func (r CheckResult) EventID() string {
+	s := fmt.Sprintf("%s%s%s", r.Event, r.Target, r.Threshold)
+	hasher := sha256.New()
+	hasher.Write([]byte(s))
+	return hex.EncodeToString(hasher.Sum(nil))
+}
+
+func (r CheckResult) String() string {
+	if r.Status == CheckOk {
+		return fmt.Sprintf("%s: %s (on sensor %s (%s) with value %s)\n", r.Status, r.Event, r.TargetName, r.Target, r.Value)
+	} else {
+		return fmt.Sprintf("%s: %s\n", r.Status, checkers[r.Event].toString(r))
+	}
+}
+
+func (box Box) RunChecks() ([]CheckResult, error) {
+	var results = []CheckResult{}
+	boxLogger := log.WithField("box", box.Id)
+
+	for _, event := range box.NotifyConf.Events {
+		for _, s := range box.Sensors {
+			// if a sensor never measured anything, thats ok. checks would fail anyway
+			if s.LastMeasurement == nil {
+				continue
+			}
+
+			if event.Target != s.Id && event.Target != eventTargetAll {
+				continue
+			}
+
+			checker := checkers[event.Type]
+			if checker.checkFunc == nil {
+				boxLogger.Warnf("ignoring unknown event type %s", event.Type)
+				continue
+			}
+
+			result, err := checker.checkFunc(event, s, box)
+			if err != nil {
+				boxLogger.Errorf("error checking event %s: %v", event.Type, err)
+			}
+
+			results = append(results, result)
+		}
+	}
+
+	return results, nil
+}

core/notifier_email.go

@@ -0,0 +1,86 @@
+package core
+
+import (
+	"errors"
+	"fmt"
+	"net/smtp"
+	"time"
+
+	"github.com/spf13/viper"
+)
+
+// box config required for the EmailNotifier (TransportConfig.Options)
+type EmailNotifier struct {
+	Recipients []string
+}
+
+func (n EmailNotifier) New(config TransportConfig) (AbstractNotifier, error) {
+	// validate transport configuration
+	// :TransportConfSourceHack @FIXME: dont get these values from viper, as the core package
+	// should be agnostic of the source of configuration!
+	requiredConf := []string{"email.user", "email.pass", "email.host", "email.port", "email.from"}
+	for _, key := range requiredConf {
+		if viper.GetString(key) == "" {
+			return nil, fmt.Errorf("Missing configuration key %s", key)
+		}
+	}
+
+	// assign configuration to the notifier after ensuring the correct type.
+	// lesson of this project: golang requires us to fuck around with type
+	// assertions, instead of providing us with proper inheritance.
+	asserted, ok := config.Options.(EmailNotifier)
+	if !ok || asserted.Recipients == nil {
+		// config did not contain valid options.
+		// first try fallback: parse result of viper is a map[string]interface{},
+		// which requires a different assertion change
+		asserted2, ok := config.Options.(map[string]interface{})
+		if ok {
+			asserted3, ok := asserted2["recipients"].([]interface{})
+			if ok {
+				asserted = EmailNotifier{Recipients: []string{}}
+				for _, rec := range asserted3 {
+					asserted.Recipients = append(asserted.Recipients, rec.(string))
+				}
+			}
+		}
+
+		if asserted.Recipients == nil {
+			return nil, errors.New("Invalid EmailNotifier options")
+		}
+	}
+
+	return EmailNotifier{
+		Recipients: asserted.Recipients,
+	}, nil
+}
+
+func (n EmailNotifier) Submit(notification Notification) error {
+	// :TransportConfSourceHack
+	auth := smtp.PlainAuth(
+		"",
+		viper.GetString("email.user"),
+		viper.GetString("email.pass"),
+		viper.GetString("email.host"),
+	)
+
+	from := viper.GetString("email.from")
+	body := fmt.Sprintf(
+		"From: openSenseMap Notifier <%s>\nDate: %s\nSubject: %s\nContent-Type: text/plain; charset=\"utf-8\"\n\n%s%s",
+		from,
+		time.Now().Format(time.RFC1123Z),
+		notification.Subject,
+		notification.Body,
+		"\n\n--\nSent automatically by osem_notify (https://github.com/noerw/osem_notify)")
+
+	// Connect to the server, authenticate, set the sender and recipient,
+	// and send the email all in one step.
+	err := smtp.SendMail(
+		fmt.Sprintf("%s:%s", viper.GetString("email.host"), viper.GetString("email.port")),
+		auth,
+		from,
+		n.Recipients,
+		[]byte(body),
+	)
+
+	return err
+}

core/notifier_slack.go

@@ -0,0 +1,73 @@
+package core
+
+import (
+	"fmt"
+	"io/ioutil"
+	"net/http"
+
+	"github.com/dghubble/sling"
+	"github.com/spf13/viper"
+)
+
+var slackClient = sling.New().Client(&http.Client{})
+
+var notificationColors = map[string]string{
+	CheckOk:  "#00ff00",
+	CheckErr: "#ff0000",
+}
+
+// slack Notifier has no configuration
+type SlackNotifier struct {
+	webhook string
+}
+
+type SlackMessage struct {
+	Text        string            `json:"text"`
+	Username    string            `json:"username,omitempty`
+	Attachments []SlackAttachment `json:"attachments,omitempty"`
+}
+
+type SlackAttachment struct {
+	Text  string `json:"text"`
+	Color string `json:"color,omitempty"`
+}
+
+func (n SlackNotifier) New(config TransportConfig) (AbstractNotifier, error) {
+	// validate transport configuration
+	// :TransportConfSourceHack
+	baseUrl := viper.GetString("slack.webhook")
+	if baseUrl == "" {
+		return nil, fmt.Errorf("Missing configuration key slack.webhook")
+	}
+
+	return SlackNotifier{
+		webhook: baseUrl,
+	}, nil
+}
+
+func (n SlackNotifier) Submit(notification Notification) error {
+	message := &SlackMessage{
+		Username:    "osem_notify box healthcheck",
+		Text:        notification.Subject,
+		Attachments: []SlackAttachment{{notification.Body, notificationColors[notification.Status]}},
+	}
+
+	req, err := slackClient.Post(n.webhook).BodyJSON(message).Request()
+	if err != nil {
+		return err
+	}
+
+	c := http.Client{}
+	res, err2 := c.Do(req)
+	if err2 != nil {
+		return err2
+	}
+
+	if res.StatusCode > 200 {
+		defer res.Body.Close()
+		body, _ := ioutil.ReadAll(res.Body)
+		return fmt.Errorf("slack webhook failed: %s", body)
+	}
+
+	return nil
+}

core/notifier_xmpp.go

@@ -0,0 +1,102 @@
+package core
+
+import (
+	"errors"
+	"fmt"
+
+	xmpp "github.com/mattn/go-xmpp"
+	"github.com/spf13/viper"
+)
+
+var xmppClient = &xmpp.Client{} // @Hacky
+
+// box config required for the XmppNotifier (TransportConfig.Options)
+type XmppNotifier struct {
+	Recipients []string
+}
+
+func (n XmppNotifier) New(config TransportConfig) (AbstractNotifier, error) {
+	// validate transport configuration
+	// :TransportConfSourceHack
+	requiredConf := []string{"xmpp.user", "xmpp.pass", "xmpp.host", "xmpp.starttls"}
+	for _, key := range requiredConf {
+		if viper.GetString(key) == "" {
+			return nil, fmt.Errorf("Missing configuration key %s", key)
+		}
+	}
+
+	// establish connection with server once, and share it accross instances
+	// @Hacky
+	if xmppClient.JID() == "" {
+		c, err := connectXmpp()
+		if err != nil {
+			return nil, err
+		}
+		xmppClient = c
+	}
+
+	// assign configuration to the notifier after ensuring the correct type.
+	// lesson of this project: golang requires us to fuck around with type
+	// assertions, instead of providing us with proper inheritance.
+	asserted, ok := config.Options.(XmppNotifier)
+	if !ok || asserted.Recipients == nil {
+		// config did not contain valid options.
+		// first try fallback: parse result of viper is a map[string]interface{},
+		// which requires a different assertion change
+		asserted2, ok := config.Options.(map[string]interface{})
+		if ok {
+			asserted3, ok := asserted2["recipients"].([]interface{})
+			if ok {
+				asserted = XmppNotifier{Recipients: []string{}}
+				for _, rec := range asserted3 {
+					asserted.Recipients = append(asserted.Recipients, rec.(string))
+				}
+			}
+		}
+
+		if asserted.Recipients == nil {
+			return nil, errors.New("Invalid XmppNotifier options")
+		}
+	}
+
+	return XmppNotifier{
+		Recipients: asserted.Recipients,
+	}, nil
+}
+
+func (n XmppNotifier) Submit(notification Notification) error {
+	if xmppClient.JID() == "" {
+		return fmt.Errorf("xmpp client not correctly initialized!")
+	}
+
+	for _, recipient := range n.Recipients {
+		_, err := xmppClient.Send(xmpp.Chat{
+			Remote:  recipient,
+			Subject: notification.Subject,
+			Text:    fmt.Sprintf("%s\n\n%s", notification.Subject, notification.Body),
+		})
+
+		if err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+func connectXmpp() (*xmpp.Client, error) {
+	// :TransportConfSourceHack
+	xmppOpts := xmpp.Options{
+		Host:     viper.GetString("xmpp.host"),
+		User:     viper.GetString("xmpp.user"),
+		Password: viper.GetString("xmpp.pass"),
+		Resource: "osem_notify",
+	}
+
+	if viper.GetBool("xmpp.starttls") {
+		xmppOpts.NoTLS = true
+		xmppOpts.StartTLS = true
+	}
+
+	return xmppOpts.NewClient()
+}

core/notifiers.go

@@ -1,99 +1,161 @@
 package core
 
 import (
-	"errors"
 	"fmt"
-	"net/smtp"
 	"strings"
 	"time"
 
-	"github.com/spf13/viper"
+	log "github.com/sirupsen/logrus"
 )
 
-var notifiers = map[string]AbstractNotifier{
+var Notifiers = map[string]AbstractNotifier{
 	"email": EmailNotifier{},
+	"slack": SlackNotifier{},
+	"xmpp":  XmppNotifier{},
 }
 
 type AbstractNotifier interface {
-	New(config interface{}) (AbstractNotifier, error)
-	ComposeNotification(box *Box, checks []CheckResult) Notification
+	New(config TransportConfig) (AbstractNotifier, error)
 	Submit(notification Notification) error
 }
 
 type Notification struct {
-	body    string
-	subject string
+	Status  string // one of CheckOk | CheckErr
+	Body    string
+	Subject string
 }
 
-// box config required for the EmailNotifier
-type EmailNotifier struct {
-	Recipients []string
+//////
+
+func (box Box) GetNotifier() (AbstractNotifier, error) {
+	return GetNotifier(&box.NotifyConf.Notifications)
 }
 
-func (n EmailNotifier) New(config interface{}) (AbstractNotifier, error) {
-	// assign configuration to the notifier after ensuring the correct type.
-	// lesson of this project: golang requires us to fuck around with type
-	// assertions, instead of providing us with proper inheritance.
-
-	asserted, ok := config.(EmailNotifier)
-	if !ok || asserted.Recipients == nil {
-		// config did not contain valid options.
-		// first try fallback: parse result of viper is a map[string]interface{},
-		// which requires a different assertion change
-		asserted2, ok := config.(map[string]interface{})
-		if ok {
-			asserted3, ok := asserted2["recipients"].([]interface{})
-			if ok {
-				asserted = EmailNotifier{Recipients: []string{}}
-				for _, rec := range asserted3 {
-					asserted.Recipients = append(asserted.Recipients, rec.(string))
-				}
+func GetNotifier(config *TransportConfig) (AbstractNotifier, error) {
+	transport := config.Transport
+
+	if transport == "" {
+		return nil, fmt.Errorf("No notification transport provided")
+	}
+
+	notifier := Notifiers[transport]
+	if notifier == nil {
+		return nil, fmt.Errorf("%s is not a supported notification transport", transport)
+	}
+
+	return notifier.New(*config)
+}
+
+func (results BoxCheckResults) SendNotifications(notifyTypes []string, useCache bool) error {
+	if useCache {
+		results = results.filterChangedFromCache()
+	}
+
+	toCheck := results.Size(notifyTypes)
+	if toCheck == 0 {
+		log.Info("No notifications due.")
+	} else {
+		log.Infof("Notifying for %v checks changing state to %v...", toCheck, notifyTypes)
+	}
+
+	errs := []string{}
+	for box, resultsBox := range results {
+		// only submit results which are errors
+		resultsDue := []CheckResult{}
+		for _, result := range resultsBox {
+			if result.HasStatus(notifyTypes) {
+				resultsDue = append(resultsDue, result)
+			}
+		}
+
+		transport := box.NotifyConf.Notifications.Transport
+		notifyLog := log.WithFields(log.Fields{
+			"boxId":     box.Id,
+			"transport": transport,
+		})
+
+		if len(resultsDue) != 0 {
+			notifier, err := box.GetNotifier()
+			if err != nil {
+				notifyLog.Error(err)
+				errs = append(errs, err.Error())
+				continue
+			}
+
+			notification := ComposeNotification(box, resultsDue)
+
+			var submitErr error
+			submitErr = notifier.Submit(notification)
+			for retry := 0; submitErr != nil && retry < 2; retry++ {
+				notifyLog.Warnf("sending notification failed (retry %v): %v", retry, submitErr)
+				time.Sleep(10 * time.Second)
+				submitErr = notifier.Submit(notification)
+			}
+			if submitErr != nil {
+				notifyLog.Error(submitErr)
+				errs = append(errs, submitErr.Error())
+				continue
 			}
 		}
 
-		if asserted.Recipients == nil {
-			return nil, errors.New("Invalid EmailNotifier options")
+		// update cache (with /all/ changed results to reset status)
+		if useCache {
+			notifyLog.Debug("updating cache")
+			updateCache(box, resultsBox)
 		}
-	}
 
-	return EmailNotifier{
-		Recipients: asserted.Recipients,
-	}, nil
-}
+		if len(resultsDue) != 0 {
+			notifyLog.Infof("Sent notification for %s via %s with %v updated issues", box.Name, transport, len(resultsDue))
+		}
+	}
 
-func (n EmailNotifier) ComposeNotification(box *Box, checks []CheckResult) Notification {
-	resultTexts := []string{}
-	for _, check := range checks {
-		resultTexts = append(resultTexts, check.String())
+	// persist changes to cache
+	if useCache {
+		err := writeCache()
+		if err != nil {
+			log.Error("could not write cache of notification results: ", err)
+			errs = append(errs, err.Error())
+		}
 	}
 
-	return Notification{
-		subject: fmt.Sprintf("Issues with your box \"%s\" on opensensemap.org!", box.Name),
-		body: fmt.Sprintf("A check at %s identified the following issue(s) with your box %s:\n\n%s\n\nYou may visit https://opensensemap.org/explore/%s for more details.\n\n--\nSent automatically by osem_notify (https://github.com/noerw/osem_notify)",
-			time.Now().Round(time.Minute), box.Name, strings.Join(resultTexts, "\n"), box.Id),
+	if len(errs) != 0 {
+		return fmt.Errorf(strings.Join(errs, "\n"))
 	}
+	return nil
 }
 
-func (n EmailNotifier) Submit(notification Notification) error {
-	auth := smtp.PlainAuth(
-		"",
-		viper.GetString("email.user"),
-		viper.GetString("email.pass"),
-		viper.GetString("email.host"),
-	)
+func ComposeNotification(box *Box, checks []CheckResult) Notification {
+	errTexts := []string{}
+	resolvedTexts := []string{}
+	for _, check := range checks {
+		if check.Status == CheckErr {
+			errTexts = append(errTexts, check.String())
+		} else {
+			resolvedTexts = append(resolvedTexts, check.String())
+		}
+	}
 
-	from := viper.GetString("email.from")
-	body := fmt.Sprintf("From: openSenseMap Notifier <%s>\nSubject: %s\nContent-Type: text/plain; charset=\"utf-8\"\n\n%s", from, notification.subject, notification.body)
-
-	// Connect to the server, authenticate, set the sender and recipient,
-	// and send the email all in one step.
-	err := smtp.SendMail(
-		fmt.Sprintf("%s:%s", viper.GetString("email.host"), viper.GetString("email.port")),
-		auth,
-		from,
-		n.Recipients,
-		[]byte(body),
+	var (
+		resolved     string
+		resolvedList string
+		errList      string
+		status       string
 	)
+	if len(resolvedTexts) != 0 {
+		resolvedList = fmt.Sprintf("Resolved issue(s):\n\n%s\n\n", strings.Join(resolvedTexts, "\n"))
+	}
+	if len(errTexts) != 0 {
+		errList = fmt.Sprintf("New issue(s):\n\n%s\n\n", strings.Join(errTexts, "\n"))
+		status = CheckErr
+	} else {
+		resolved = "resolved "
+		status = CheckOk
+	}
 
-	return err
+	return Notification{
+		Status:  status,
+		Subject: fmt.Sprintf("Issues %swith your box \"%s\" on opensensemap.org!", resolved, box.Name),
+		Body: fmt.Sprintf("A check at %s identified the following updates for your box \"%s\":\n\n%s%sYou may visit https://opensensemap.org/explore/%s for more details.",
+			time.Now().Round(time.Minute), box.Name, errList, resolvedList, box.Id),
+	}
 }

core/osem_api.go

@@ -0,0 +1,96 @@
+package core
+
+import (
+	"errors"
+	"net/http"
+	"time"
+
+	"github.com/dghubble/sling"
+)
+
+type OsemError struct {
+	Code    string `json:"code"`
+	Message string `json:"message"`
+}
+
+type BoxFilters struct {
+	Date       string `url:"date,omitempty"`
+	Exposure   string `url:"exposure,omitempty"`
+	Grouptag   string `url:"grouptag,omitempty"`
+	Model      string `url:"model,omitempty"`
+	Phenomenon string `url:"phenomenon,omitempty"`
+}
+
+type OsemClient struct {
+	sling *sling.Sling
+}
+
+func NewOsemClient(endpoint string) *OsemClient {
+	return &OsemClient{
+		sling: sling.New().Client(&http.Client{}).Base(endpoint),
+	}
+}
+
+func (client *OsemClient) GetBox(boxId string) (*Box, error) {
+	box := &Box{}
+	fail := &OsemError{}
+	_, err := client.sling.New().Path("boxes/").Path(boxId).Receive(box, fail)
+	if err != nil {
+		return nil, err
+	}
+	if fail.Message != "" {
+		return box, errors.New("could not fetch box: " + fail.Message)
+	}
+	return box, nil
+}
+
+func (client *OsemClient) GetAllBoxes(params BoxFilters) (*[]BoxMinimal, error) {
+	boxes := &[]BoxMinimal{}
+	fail := &OsemError{}
+	_, err := client.sling.New().Path("boxes?minimal=true").QueryStruct(params).Receive(boxes, fail)
+	if err != nil {
+		return nil, err
+	}
+	if fail.Message != "" {
+		return boxes, errors.New("could not fetch boxes: " + fail.Message)
+	}
+	return boxes, nil
+}
+
+type NotifyEvent struct {
+	Type      string `json:"type"`
+	Target    string `json:"target"`
+	Threshold string `json:"threshold"`
+}
+
+type TransportConfig struct {
+	Transport string      `json:"transport"`
+	Options   interface{} `json:"options"`
+}
+
+type NotifyConfig struct {
+	Notifications TransportConfig `json:"notifications"`
+	Events        []NotifyEvent   `json:"events"`
+}
+
+type Sensor struct {
+	Id              string `json:"_id"`
+	Phenomenon      string `json:"title"`
+	Type            string `json:"sensorType"`
+	LastMeasurement *struct {
+		Value string    `json:"value"`
+		Date  time.Time `json:"createdAt"`
+	} `json:"lastMeasurement"`
+}
+
+type Box struct {
+	Id         string        `json:"_id"`
+	Name       string        `json:"name"`
+	Sensors    []Sensor      `json:"sensors"`
+	NotifyConf *NotifyConfig `json:"healthcheck"`
+}
+
+type BoxMinimal struct {
+	Id   string `json:"_id"`
+	Name string `json:"name"`
+}

doc/osem_notify.md

@@ -1,29 +0,0 @@
-## osem_notify
-
-Root command displaying help
-
-### Synopsis
-
-Run healthchecks and send notifications for boxes on opensensemap.org
-
-```
-osem_notify [flags]
-```
-
-### Options
-
-```
-  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
-  -d, --debug              enable verbose logging
-  -h, --help               help for osem_notify
-  -l, --logformat string   log format, can be plain or json (default "plain")
-  -n, --notify             if set, will send out notifications.
-                           Otherwise results are printed to stdout only
-```
-
-### SEE ALSO
-
-* [osem_notify check](osem_notify_check.md)	 - One-off check for events on boxes
-* [osem_notify watch](osem_notify_watch.md)	 - Watch boxes for events at an interval
-
-###### Auto generated by spf13/cobra on 24-Jun-2018

doc/osem_notify_check.md

@@ -1,30 +0,0 @@
-## osem_notify check
-
-One-off check for events on boxes
-
-### Synopsis
-
-One-off check for events on boxes
-
-### Options
-
-```
-  -h, --help   help for check
-```
-
-### Options inherited from parent commands
-
-```
-  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
-  -d, --debug              enable verbose logging
-  -l, --logformat string   log format, can be plain or json (default "plain")
-  -n, --notify             if set, will send out notifications.
-                           Otherwise results are printed to stdout only
-```
-
-### SEE ALSO
-
-* [osem_notify](osem_notify.md)	 - Root command displaying help
-* [osem_notify check boxes](osem_notify_check_boxes.md)	 - one-off check on one or more box IDs
-
-###### Auto generated by spf13/cobra on 24-Jun-2018

doc/osem_notify_check_boxes.md

@@ -1,33 +0,0 @@
-## osem_notify check boxes
-
-one-off check on one or more box IDs
-
-### Synopsis
-
-specify box IDs to check them for events
-
-```
-osem_notify check boxes <boxId> [...<boxIds>] [flags]
-```
-
-### Options
-
-```
-  -h, --help   help for boxes
-```
-
-### Options inherited from parent commands
-
-```
-  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
-  -d, --debug              enable verbose logging
-  -l, --logformat string   log format, can be plain or json (default "plain")
-  -n, --notify             if set, will send out notifications.
-                           Otherwise results are printed to stdout only
-```
-
-### SEE ALSO
-
-* [osem_notify check](osem_notify_check.md)	 - One-off check for events on boxes
-
-###### Auto generated by spf13/cobra on 24-Jun-2018

doc/osem_notify_watch.md

@@ -1,31 +0,0 @@
-## osem_notify watch
-
-Watch boxes for events at an interval
-
-### Synopsis
-
-Watch boxes for events at an interval
-
-### Options
-
-```
-  -h, --help           help for watch
-  -i, --interval int   interval to run checks in minutes (default 15)
-```
-
-### Options inherited from parent commands
-
-```
-  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
-  -d, --debug              enable verbose logging
-  -l, --logformat string   log format, can be plain or json (default "plain")
-  -n, --notify             if set, will send out notifications.
-                           Otherwise results are printed to stdout only
-```
-
-### SEE ALSO
-
-* [osem_notify](osem_notify.md)	 - Root command displaying help
-* [osem_notify watch boxes](osem_notify_watch_boxes.md)	 - watch a list of box IDs for events
-
-###### Auto generated by spf13/cobra on 24-Jun-2018

doc/osem_notify_watch_boxes.md

@@ -1,34 +0,0 @@
-## osem_notify watch boxes
-
-watch a list of box IDs for events
-
-### Synopsis
-
-specify box IDs to watch them for events
-
-```
-osem_notify watch boxes <boxId> [...<boxIds>] [flags]
-```
-
-### Options
-
-```
-  -h, --help   help for boxes
-```
-
-### Options inherited from parent commands
-
-```
-  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
-  -d, --debug              enable verbose logging
-  -i, --interval int       interval to run checks in minutes (default 15)
-  -l, --logformat string   log format, can be plain or json (default "plain")
-  -n, --notify             if set, will send out notifications.
-                           Otherwise results are printed to stdout only
-```
-
-### SEE ALSO
-
-* [osem_notify watch](osem_notify_watch.md)	 - Watch boxes for events at an interval
-
-###### Auto generated by spf13/cobra on 24-Jun-2018

docs/_config.yml

@@ -0,0 +1,6 @@
+theme: jekyll-theme-hacker
+plugins:
+  - jekyll-redirect-from
+whitelist:
+  - jekyll-redirect-from
+

docs/index.md

@@ -0,0 +1,4 @@
+---
+redirect_to:
+  - "osem_notify"
+---

docs/osem_notify.md

@@ -0,0 +1,40 @@
+## osem_notify
+
+Root command displaying help
+
+### Synopsis
+
+Run healthchecks and send notifications for boxes on opensensemap.org
+
+```
+osem_notify [flags]
+```
+
+### Options
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -h, --help               help for osem_notify
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify check](osem_notify_check.md)	 - One-off check for events on boxes
+* [osem_notify debug](osem_notify_debug.md)	 - Run some debugging checks on osem_notify itself
+* [osem_notify version](osem_notify_version.md)	 - Get build and version information
+* [osem_notify watch](osem_notify_watch.md)	 - Watch boxes for events at an interval
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_check.md

@@ -0,0 +1,40 @@
+## osem_notify check
+
+One-off check for events on boxes
+
+### Synopsis
+
+One-off check for events on boxes
+
+### Options
+
+```
+  -h, --help   help for check
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify](osem_notify.md)	 - Root command displaying help
+* [osem_notify check all](osem_notify_check_all.md)	 - one-off check on all boxes registered on the opensensemap instance
+* [osem_notify check boxes](osem_notify_check_boxes.md)	 - one-off check on one or more box IDs
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_check_all.md

@@ -0,0 +1,42 @@
+## osem_notify check all
+
+one-off check on all boxes registered on the opensensemap instance
+
+### Synopsis
+
+one-off check on all boxes registered on the opensensemap instance
+
+```
+osem_notify check all [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for all
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify check](osem_notify_check.md)	 - One-off check for events on boxes
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_check_boxes.md

@@ -0,0 +1,42 @@
+## osem_notify check boxes
+
+one-off check on one or more box IDs
+
+### Synopsis
+
+specify box IDs to check them for events
+
+```
+osem_notify check boxes <boxId> [...<boxIds>] [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for boxes
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify check](osem_notify_check.md)	 - One-off check for events on boxes
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_debug.md

@@ -0,0 +1,44 @@
+## osem_notify debug
+
+Run some debugging checks on osem_notify itself
+
+### Synopsis
+
+osem_notify debug <feature> tests the functionality of the given feature
+
+```
+osem_notify debug [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for debug
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify](osem_notify.md)	 - Root command displaying help
+* [osem_notify debug cache](osem_notify_debug_cache.md)	 - Print or clear the notifications cache
+* [osem_notify debug notifications](osem_notify_debug_notifications.md)	 - Verify that notifications are working
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_debug_cache.md

@@ -0,0 +1,43 @@
+## osem_notify debug cache
+
+Print or clear the notifications cache
+
+### Synopsis
+
+osem_notify debug cache prints the contents of the notifications cache
+
+```
+osem_notify debug cache [flags]
+```
+
+### Options
+
+```
+      --clear   reset the notifications cache
+  -h, --help    help for cache
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify debug](osem_notify_debug.md)	 - Run some debugging checks on osem_notify itself
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_debug_notifications.md

@@ -0,0 +1,43 @@
+## osem_notify debug notifications
+
+Verify that notifications are working
+
+### Synopsis
+
+osem_notify debug notifications sends a test notification according
+to healthchecks.default.notifications.options as defined in the config file
+
+```
+osem_notify debug notifications [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for notifications
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify debug](osem_notify_debug.md)	 - Run some debugging checks on osem_notify itself
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_version.md

@@ -0,0 +1,42 @@
+## osem_notify version
+
+Get build and version information
+
+### Synopsis
+
+osem_notify version returns its build and version information
+
+```
+osem_notify version [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for version
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify](osem_notify.md)	 - Root command displaying help
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_watch.md

@@ -0,0 +1,41 @@
+## osem_notify watch
+
+Watch boxes for events at an interval
+
+### Synopsis
+
+Watch boxes for events at an interval
+
+### Options
+
+```
+  -h, --help           help for watch
+  -i, --interval int   interval to run checks in minutes (default 30)
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify](osem_notify.md)	 - Root command displaying help
+* [osem_notify watch all](osem_notify_watch_all.md)	 - watch all boxes registered on the map
+* [osem_notify watch boxes](osem_notify_watch_boxes.md)	 - watch a list of box IDs for events
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_watch_all.md

@@ -0,0 +1,43 @@
+## osem_notify watch all
+
+watch all boxes registered on the map
+
+### Synopsis
+
+watch all boxes registered on the map
+
+```
+osem_notify watch all [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for all
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -i, --interval int       interval to run checks in minutes (default 30)
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify watch](osem_notify_watch.md)	 - Watch boxes for events at an interval
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

docs/osem_notify_watch_boxes.md

@@ -0,0 +1,43 @@
+## osem_notify watch boxes
+
+watch a list of box IDs for events
+
+### Synopsis
+
+specify box IDs to watch them for events
+
+```
+osem_notify watch boxes <boxId> [...<boxIds>] [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for boxes
+```
+
+### Options inherited from parent commands
+
+```
+  -a, --api string         openSenseMap API to query against (default "https://api.opensensemap.org")
+  -c, --config string      path to config file (default $HOME/.osem_notify.yml)
+  -d, --debug              enable verbose logging
+  -i, --interval int       interval to run checks in minutes (default 30)
+  -l, --logformat string   log format, can be plain or json (default "plain")
+      --no-cache           send all notifications, ignoring results from previous runs. also don't update the cache.
+  -n, --notify string      If set, will send out notifications for the specified type of check result,
+                           otherwise results are printed to stdout only.
+                           Allowed values are "all", "error", "ok".
+                           You might want to run 'osem_notify debug notifications' first to verify everything works.
+                           
+                           Notifications for failing checks are sent only once, and then cached until the issue got
+                           resolved, unless --no-cache is set.
+                           To clear the cache, run 'osem_notify debug cache --clear'.
+                           
+```
+
+### SEE ALSO
+
+* [osem_notify watch](osem_notify_watch.md)	 - Watch boxes for events at an interval
+
+###### Auto generated by spf13/cobra on 10-Feb-2019

utils/config.go

@@ -0,0 +1,78 @@
+package utils
+
+import (
+	"os"
+	"path"
+	"strconv"
+	"strings"
+
+	log "github.com/sirupsen/logrus"
+	"github.com/spf13/viper"
+)
+
+/**
+ * config file handling, as it is kinda broken in spf13/viper
+ * mostly copied from https://github.com/TheThingsNetwork/ttn/blob/f623a6a/ttnctl/util/config.go
+ */
+
+// GetConfigFile returns the location of the configuration file.
+// It checks the following (in this order):
+// the --config flag
+// $XDG_CONFIG_HOME/osem_notify/config.yml (if $XDG_CONFIG_HOME is set)
+// $HOME/.osem_notify.yml
+func GetConfigFile(name string) string {
+	flag := viper.GetString("config")
+
+	xdg := os.Getenv("XDG_CONFIG_HOME")
+	if xdg != "" {
+		xdg = path.Join(xdg, name, "config.yml")
+	}
+
+	home := os.Getenv("HOME")
+	homeyml := ""
+	homeyaml := ""
+
+	if home != "" {
+		homeyml = path.Join(home, "."+name+".yml")
+		homeyaml = path.Join(home, "."+name+".yaml")
+	}
+
+	tryFiles := []string{
+		flag,
+		xdg,
+		homeyml,
+		homeyaml,
+	}
+
+	// find a file that exists, and use that
+	for _, file := range tryFiles {
+		if file != "" {
+			if _, err := os.Stat(file); err == nil {
+				return file
+			}
+		}
+	}
+
+	// no file found, set up correct fallback
+	if os.Getenv("XDG_CONFIG_HOME") != "" {
+		return xdg
+	} else {
+		return homeyml
+	}
+}
+
+func PrintConfig() {
+	log.Debug("Using config:")
+	printKV("config file", viper.ConfigFileUsed())
+	for key, val := range viper.AllSettings() {
+		printKV(key, val)
+	}
+}
+
+func printKV(key, val interface{}) {
+	log.Debugf("%20s: %v", key, val)
+}
+
+func ParseFloat(val string) (float64, error) {
+	return strconv.ParseFloat(strings.TrimSpace(val), 64)
+}