HyperDX Dashboard Provisioner#190
Conversation
🦋 Changeset detectedLatest commit: a437052 The changes in this PR will be included in the next version bump. Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
|
|
Implements feature request #170 |
ZeynelKoca
force-pushed
the
inject-json-dashboards
branch
from
0243f23 to
752d6a5
Compare
|
@dhable Can we please look at this PR and see how we can progress it ? |
ZeynelKoca
force-pushed
the
inject-json-dashboards
branch
2 times, most recently
from
7d182d9 to
64af4b1
Compare
|
As per PR description, I redesigned the initial implementation to handle upsert behavior natively through hyperdx (same pattern as Grafana). This PR is dependent on hyperdxio/hyperdx#1962 |
ZeynelKoca
force-pushed
the
inject-json-dashboards
branch
3 times, most recently
from
04bbee1 to
5714065
Compare
dhable
left a comment
dhable
left a comment
There was a problem hiding this comment.
Thank you for following up the feature change with a helm change as well. Great to keep these two up-to-date.
I think the need to provision RBAC roles and scan for tagged config maps is more than our typical users would expect from an application install. Using the default ConfigMap functionality to mount the dashboard data as a file would avoid the need for an additional service account, RBAC, etc.
| dashboards: | ||
| enabled: false | ||
| # Image for the k8s-sidecar that watches for labeled ConfigMaps | ||
| sidecarImage: "kiwigrid/k8s-sidecar:2.5.0" |
There was a problem hiding this comment.
What is the motivation for using a sidecar to store the ConfigMap content as a file vs using the built in ability of ConfigMaps to be used as a file? I am slightly concerned about expanding 3rd party dependencies and increasing the security attack surface area of a deployment.
There was a problem hiding this comment.
The use case is: the team managing ClickStack deploys and manages the observability platform independently from application teams. Each application team manages their own HyperDX dashboards as ConfigMaps in their own namespace, deployed via their own Helm charts- GitOps style
Without the sidecar, the ClickStack team would need to track every application team's dashboards and redeploy ClickStack every time a dashboard is added or changed. The sidecar dynamically discovers labeled ConfigMaps across namespaces (hence the RBAC)
This pattern is directly taken from Grafana's Helm chart, which uses the same kiwigrid/k8s-sidecar image for the same purpose (see https://github.com/grafana-community/helm-charts/blob/main/charts/grafana/values.yaml)
| imagePullSecrets: | ||
| {{- toYaml .Values.global.imagePullSecrets | nindent 8 }} | ||
| {{- end }} | ||
| {{- if .Values.hyperdx.dashboards.enabled }} |
There was a problem hiding this comment.
We merged in a SA change recently and if that field is set and dashboards are enabled, the resulting template ends up with two serviceAccountName entries. We need any change to be compatible with user defined SA's in order to work within AWS hosted clusters effectively.
There was a problem hiding this comment.
Good catch, I rebased onto main and changed SA handling to support user defined SAs
k8s-sidecar watches ConfigMaps labeled "hyperdx.io/dashboard: true" across all namespaces and writes dashboard JSON to a shared volume. HyperDX reads and upserts them natively via file-based provisioner. Requires hyperdxio/hyperdx#1962
ZeynelKoca
force-pushed
the
inject-json-dashboards
branch
from
bd006b9 to
a437052
Compare
No worries, the dashboard provisioning would be really nice for us as we're planning to go into production with clickstack soon and implementing missing features is faster than creating an issue 😄 Ready for another review! |
|
@dhable Gentle bump. Happy to hear your thoughts on the implementation |
|
@dhable it would be great if this feature could be released! Very handy. |
## Summary
Add a `provision-dashboards` task that reads `.json` files from a directory and upserts dashboards into MongoDB, following the existing task system pattern (same as `check-alerts`).
Provisioned dashboards are flagged with `provisioned: true` so they never overwrite user-created dashboards with the same name. Files are validated against `DashboardWithoutIdSchema`. Removing a file does not delete the dashboard (safe by default, same as Grafana). The task is deployment-agnostic: it reads from a directory, regardless of how files get there.
When `DASHBOARD_PROVISIONER_DIR` is set, `entry.prod.sh` automatically starts the task as an additional process alongside the API, App, and check-alerts.
**Note:** Users can currently edit provisioned dashboards through the UI, but changes will be overwritten on the next sync cycle. Grafana handles this by blocking saves on provisioned dashboards. Adding a similar guard would be a good follow-up to improve UX.
| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `DASHBOARD_PROVISIONER_DIR` | Yes | | Directory to read `.json` files from |
| `DASHBOARD_PROVISIONER_TEAM_ID` | No* | | Scope to a specific team ID |
| `DASHBOARD_PROVISIONER_ALL_TEAMS` | No* | `false` | Set to `true` to provision to all teams |
\*One of `DASHBOARD_PROVISIONER_TEAM_ID` or `DASHBOARD_PROVISIONER_ALL_TEAMS=true` is required.
### How to test locally or on Vercel
1. Create a directory with a dashboard JSON file:
```bash
mkdir /tmp/dashboards
echo '{"name":"Test Dashboard","tiles":[],"tags":[]}' > /tmp/dashboards/test.json
2. Run the task:
DASHBOARD_PROVISIONER_DIR=/tmp/dashboards DASHBOARD_PROVISIONER_ALL_TEAMS=true \
./packages/api/bin/hyperdx task provision-dashboards
3. Verify the dashboard appears in the UI
4. Modify the JSON file, run again, verify it updates
5. Delete the JSON file, run again, verify the dashboard persists
References
- Related PRs: ClickHouse/ClickStack-helm-charts#190
5 participants
Summary
hyperdx.io/dashboard: "true"across all namespacesDesign decisions
Dependencies