See what broke. Understand why. Know what to do next. 👀🧠⚡
Website · Docs · Discord · Issues
kwatch is an open-source Kubernetes incident monitor. It turns failures into clear alerts that explain what broke, why it happened, and what to check next.
It runs inside your own cluster. There is no hosted account, agent platform, or required observability stack.
Kubernetes can tell you that a Pod is failing. kwatch adds the context needed to act:
| Kubernetes signal | What kwatch adds |
|---|---|
CrashLoopBackOff |
Reason, logs, events, owner, and next step |
Pending |
Scheduling clues and unschedulable duration |
| Unhealthy node | Affected workloads and dependency impact |
| Many related failures | One grouped incident instead of alert noise |
Alerts can include the workload, owner, namespace, Pod, container, node, recent logs, Kubernetes events, related dependencies, and a suggested action.
The supported path is the interactive kwatch.sh manager. It:
- Lets you choose the kubeconfig context without changing your current one.
- Asks where to send alerts.
- Stores credentials in a Kubernetes Secret.
- Installs the CRD and hardened kwatch workload.
- Waits for the deployment to become ready and checks its security posture.
Run it with no version parameters:
/bin/bash -c "$(curl -fsSL https://kwatch.dev/kwatch.sh)"You need Bash, curl, kubectl, and permission to install the required
namespace-scoped and cluster-scoped resources.
The manager selects the latest stable release by default. During installation and upgrade it lets you choose the newest published release candidate interactively when one is available; no version parameter is required.
Use the manager again after installation to configure alerts, change settings,
upgrade, check status, or uninstall kwatch. Do not apply deploy.yaml or
config.yaml manually; that bypasses guided Secret handling and verification.
🚨 OOMKilled — production / orders-api
Pod: orders-api-7ffc9d4f9-x9p4t
Node: worker-3 · severity: high
💡 Cause: the container exceeded its 512Mi memory limit.
➡️ Next step: increase limits.memory or reduce memory usage.
📄 Recent logs and Kubernetes events are included.
Core monitors are enabled by default; optional integrations are opt-in:
| Area | Examples |
|---|---|
| Pods and containers | Crashes, OOM kills, restarts, and readiness |
| Scheduling | Pending Pods, unschedulable workloads, and delay |
| Workloads | Deployments, StatefulSets, DaemonSets, Jobs, CronJobs, and PDBs |
| Infrastructure | Nodes, resource pressure, disk, and inode usage |
| Storage | PVC usage and persistent-volume failures |
| Networking | Services, Ingress, webhooks, and NetworkPolicies |
| Scaling | HPA and cluster-autoscaler signals |
| Platform health | Control plane, kubelet telemetry, and cluster resources |
| Security and policy | TLS expiry, RBAC, admission, and Pod Security findings |
Heartbeat notifications, Metrics Server usage, TLS certificate monitoring, and
active probes are opt-in. Generic custom-resource status checks use
clusterResourceMonitor (enabled by default) and run only for resources the
ServiceAccount can list and watch. The KwatchConfig overlay is enabled by
Helm and the interactive installer, while the standalone binary defaults
crd.enabled to false. See the
configuration reference for
defaults, permissions, and thresholds.
kwatch supports 56 notification integrations, including Slack, Discord, Microsoft Teams, Google Chat, Telegram, email, PagerDuty, Opsgenie, Mattermost, Rocket.Chat, Matrix, webhooks, Jira, and Datadog.
Configure one or more channels under alert:. The
alert channel guide has setup examples,
routing, retries, fallbacks, and the complete provider list.
Provider credentials, diagnostic tokens, and heartbeat URLs must be mounted
from a Kubernetes Secret. Use an exact file reference in config.yaml:
app:
clusterName: production
alert:
slack:
webhook: "${file:/config/slack-webhook}"Plain credentials and ${ENV_VAR} substitutions are rejected for sensitive
fields. The manager creates the Secret and writes only file references to the
configuration.
| Setting | Use it to... |
|---|---|
namespaces |
Watch only selected namespaces |
reasons |
Include or exclude alert reasons |
silences |
Suppress known, intentional failures, including matching Event messages |
includeLogs / includeEvents |
Add Kubernetes context to alerts |
smartGrouping |
Combine related symptoms |
correlation |
Track, resolve, cool down, and re-notify incidents |
app.clusterName |
Identify the cluster in every alert |
Run kwatch lint before restarting with an edited configuration. Add --check
to verify credentials for providers that support checks.
Run the same manager command again after installation:
install Install kwatch
configure-alert Change the notification destination
configure Change monitors, thresholds, and silences
upgrade Upgrade to stable or choose an available RC
status Show deployment and manager state
features Show the capabilities of the installed release
uninstall Remove the workload and notification Secret
The manager backs up configuration before upgrades and keeps the KwatchConfig,
backups, namespace, and CRD during uninstall so a future reinstall can recover
the existing setup.
The container also includes a small CLI for operators and automation:
| Command | Purpose |
|---|---|
kwatch --version |
Print the short version |
kwatch version --json |
Print build information as JSON |
kwatch lint --strict |
Validate config and reject unknown fields |
kwatch lint --check |
Validate config and provider checks |
kwatch replay --dry-run < events.jsonl |
Preview replay without sending |
kwatch replay sends real notifications by default. Use --dry-run when you
only want to preview the result.
kwatch is an alerting and diagnosis layer. It is not a metrics database, log store, dashboard, or query language. Use Prometheus/Grafana for long-term metrics and Loki for log search; use kwatch when something changes and you need a useful explanation quickly.
- Repository documentation index
- Getting started
kwatch.shmanager- Configuration
- Alert channels
- CLI commands
- Kubernetes coverage
- Architecture
- Release integrity
Bug report, new idea, or documentation improvement? Open an issue or pull request.
kwatch is available under the MIT License.
