A Kubernetes operator for running synthetic checks as pods. Works great with Prometheus!
khcheck
khcheck
client is simple! :)If you are using Go, we have an easy to use client package: found here.
example code:
package main
import (
"github.com/kuberhealthy/kuberhealthy/v2/pkg/checks/external/checkclient"
)
func main() {
ok := doCheckStuff()
if !ok {
checkclient.ReportFailure([]string{"Test has failed!"})
return
}
checkclient.ReportSuccess()
}
An example check with working Dockerfile is available to use as an example here.
If you would like to write a check in JavaScript, there is a sample client for checks found here.
Please see the example check under the same folder with a Dockerfile for reference.
The kuberhealthy NPM package is conformant with the reference sample syntax but also supports async/await as well as arbitrary host and port.
npm i --save kuberhealthy
example code:
const kh = require('kuberhealthy')
const report = async () => {
let ok = await doCheckStuff()
if (ok) {
await kh.ReportSuccess()
} else {
await kh.ReportFailure()
}
}
report()
NOTE: KH_REPORTING_URL must be set in your env. This is usually done automatically if running as ‘khcheck’ on kubernetes.
Your check only needs to do a few things:
KH_REPORTING_URL
environment variable.POST
to the KH_REPORTING_URL
with the following JSON body:KH_CHECK_RUN_DEADLINE
environment variable. If KH_CHECK_RUN_DEADLINE
is not respected, your check may run into a 400
error when reporting its state to Kuberhealthy. This deadline is calculated by the timeout:
setting in the khcheck
resource.{
"Errors": [
"Error 1 here",
"Error 2 here"
],
"OK": false
}
Never send
"OK": true
ifErrors
has values or you will be given a400
return code.
Simply build your program into a container, docker push
it to somewhere your cluster has access and craft a khcheck
resource to enable it in your cluster where Kuberhealthy is installed.
Clients outside of Go can be found in the clients directory.
The following environment variables are injected into every checker pod that Kuberhealthy runs. When writing your checker code, you can depend on these environment variables always being available to you, even if you do not specify them in your khcheck
spec.
KH_REPORTING_URL: The Kuberhealthy URL to send POST requests to for check statuses.
KH_CHECK_RUN_DEADLINE: The Kuberhealthy deadline for checks as calculated by the check timeout given in Unix.
KH_RUN_UUID: The UUID of the check run. This must be sent back as the header 'kh-run-uuid' when status is reported to KH_REPORTING_URL. The Go checkClient package does this automatically.
KH_POD_NAMESPACE: The namespace of the checker pod.
khcheck
ResourceEvery check needs a khcheck
to enable and configure it. As soon as this resource is applied to the cluster, Kuberhealthy will begin running your check. Whenever you make a change, Kuberhealthy will automatically re-load the check and restart any checks currently in progress gracefully.
Here is a minimal khcheck
resource to start hacking with:
apiVersion: comcast.github.io/v1
kind: KuberhealthyCheck
metadata:
name: kh-test-check
spec:
runInterval: 30s # The interval that Kuberhealthy will run your check on
timeout: 2m # After this much time, Kuberhealthy will kill your check and consider it "failed"
podSpec: # The exact pod spec that will run. All normal pod spec is valid here.
containers:
- env: # Environment variables are optional but a recommended way to configure check behavior
- name: MY_OPTION_ENV_VAR
value: "option_setting_here"
image: quay.io/comcast/test-check:latest # The image of the check you just pushed
imagePullPolicy: Always # During check development, it helps to set this to 'Always' to prevent on-node image caching.
name: main
That’s it! As soon as this khcheck
is applied, Kuberhealthy will begin running your check, serving prometheus metrics for it, and displaying status JSON on the status page.
You can see a list of checks that others have written on the check registry. If you have a check that may be useful to others and want to contribute, consider adding it to the registry! Just fork this repository and send a PR. This is made easy by simply checking the Edit
pencil on the check registry page.