Alerts can be sent from jobs and feature group validations. This is done by creating job alerts or feature group alerts for specific jobs or feature groups. Alternatively alerts can be created for all jobs or feature group validations in a project.
In order to send alerts, routes and receivers need to be configured. This is done from the project settings page.
Receiver is a named configuration that supports notification methods such as email, slack, PagerDuty, Pushover, Webhook, Victorops, Wechat or OpsGenie. The configurations are given as a list of json objects. Allowable values for the different configuration json objects are given below.
[{"to": "example@hopsworks.ai", ...}, {...}]
For a complete list of available values for email config see email config.
Note
The links above contain the yaml formatted values.
When converting the keys to json do not forget to replace Snake Case (underscores) to Camel Case.
The figure below shows existing receivers on the right and a form for creating new receivers.
Routes are used to match alerts to receivers. A receiver needs to exist in order to create a route. Alerts are sent to receivers if they match all the equality (match) or regular expression matchers (matchRe). Match and matchRe are given as a key value pair as shown in the figure below.
Silences can be used to mute alerts for a given time. A silence is configured based on matchers, just like the routing tree. Incoming alerts are checked whether they match all the equality or regular expression matchers of an active silence. If they do, no notifications will be sent out for that alert.
Silences can be created from the project setting page.
Note
Deleting a silence will only set the status to expired. Expired silences will be automatically garbage collected after two weeks.
Service alerts can be created for jobs and feature group validations. After creating an alert a test alert can be sent by clicking on the paper airplane in the action column.
Global alerts are sent to receivers configured by platform administrators. While project alerts are sent to the project’s receivers.
Alerts can be created for a specific job by going to the jobs UI and clicking on the Alers menu of the job you want to create alerts for.
This will open a modal shown below.
Global alerts are sent to receivers configured by platform administrators. While project alerts are sent to the project’s receivers.
Feature group validation can also send alerts. To configure alerts for a specific feature group go to the feature group’s details page and click on the validation alerts accordion.
Global alerts are sent to receivers configured by platform administrators. While project alerts are sent to the project’s receivers.
All active alerts can be seen in the project settings page. This is especially useful if an alert fails to reach the intended receiver. By expanding an active alert you can see if it is matching the correct receiver and if it is being muted by a silence.
The figure below shows an alert that matched receiver ‘demo_spark_meb1000’ but is suppressed by a silence with id ‘6764906c-f7e4-4b1f-9b58-bb491c98f3c9’.