Advancing Dashboarding in Grafana
We design dashboards that speak to both engineering teams and business leadership. In this article we share practical patterns and recipes to:
- Integrate and correlate data from multiple sources (APIs, Splunk indexes, Datadog, Dynatrace, InfluxDB, etc.) in the same Grafana dashboard or even the same panel.
- Use advanced plugins such as the Business Charts suite to create tailored visualizations and business-friendly representations.
- Deliver reusable, safe dashboard artifacts (JSON templates, dashboards-as-code) and governance around metrics for cross-functional consumption.
Why correlate multiple sources in the same dashboard?
Systems rarely live in a single observability tool. Infrastructure metrics may live in Prometheus, business metrics in an analytics DB, APM traces in Datadog/Dynatrace, and logs or security events in Splunk. Combining these signals into a single view lets us quickly answer operational and business questions such as:
- Are recent latency spikes in a service correlated with a specific deployment (CI/CD) or a rise in error rates reported in logs?
- Did an increase in support tickets (CRM) match an outage window detected by monitoring?
- How is a new feature adoption (business event) impacting infrastructure costs?
Grafana supports mixed data sources and provides transformations to join and reshape queries — this enables correlation across systems without forcing a single backend. :contentReference[oaicite:0]{index=0}
Common patterns for multi-source dashboarding
1) Mixed-Panel pattern (multiple queries, single panel)
Use Grafana's Mixed datasource to add multiple queries (each query can target a different datasource) to the same panel. This is ideal for aligned time-series charts or tables where each series originates in a different system. Typical uses:
- Plot application latency from APM (e.g., Datadog) with infrastructure CPU from Prometheus.
- Show business KPIs from an analytics DB next to error rates from logs.
Advice: normalize time ranges and sampling resolutions, and use transformations (join, outer join, merge) to align series. Grafana docs and community posts explain how to work with multiple datasources in practice. :contentReference[oaicite:1]{index=1}
2) API-aggregation pattern (proxy / mediator)
When complex correlation or custom business logic is required (e.g., enrich APM traces with CRM events), we recommend an API-aggregation layer:
- We build a small service that fetches data from multiple vendor APIs (Datadog, Dynatrace, Splunk, custom timeseries DBs).
- That service merges, normalizes and exposes a single JSON/Prometheus endpoint tailored to the dashboard’s needs.
- Grafana consumes that endpoint using the JSON API datasource or Prometheus remote write (depending on the format).
This pattern centralizes transformations that may be difficult to express inside Grafana and makes dashboards reproducible and testable.
Example: simple Node.js aggregator (pseudo-code)
// This is illustrative. Replace with robust code for auth, retries, caching.
const express = require('express');
const fetch = require('node-fetch'); // or axios
const app = express();
async function fetchDatadog() {
// call Datadog API (metrics/queries)
return fetch('https://api.datadoghq.com/api/v1/query?from=...&to=...').then(r => r.json());
}
async function fetchSplunk() {
// call Splunk (or Splunk HTTP Event Collector)
return fetch('https://splunk.example/api/search?...').then(r => r.json());
}
app.get('/api/combined', async (req, res) => {
const [dd, sp] = await Promise.all([fetchDatadog(), fetchSplunk()]);
// normalize and merge by timestamp
const merged = mergeByTimestamp(dd.series, sp.series);
res.json({ series: merged });
});
app.listen(8080);
Grafana can consume /api/combined with a JSON datasource plugin (for example, the JSON API or SimpleJSON plugin) and visualize the merged series in a panel. Using an aggregator also lets us apply business logic (e.g., map user IDs to feature flags) before presenting it to executives.
3) Transformations & joins inside Grafana
If an aggregator is not desirable, Grafana's Transformations let you join query results at the panel level (outer join, merge, labels). This works well for tables or when you have matching keys across datasources. Be mindful of cardinality and performance when joining large tables in the browser. :contentReference[oaicite:2]{index=2}
Business-facing visualizations & Business Charts plugin
Technical teams and business stakeholders have different needs: engineers need high-cardinality, fine-grained traces and error windows; leadership wants clear KPIs, trends, and executive summaries. To bridge this gap we:
- Design "two-layer" dashboards: a top layer with business KPIs (one-number metrics, trend-sparklines, forecasts) and a second layer with drilldowns for engineering details.
- Use dedicated business visualization plugins when standard Grafana panels are insufficient.
One widely-used suite is Volkov Labs' Business Charts (an Apache ECharts-based panel) and the broader Business Suite (Business Table, Business Input). These plugins enable advanced, code-driven charts and custom rendering for business dashboards. Their documentation and examples are a great resource. :contentReference[oaicite:3]{index=3}
Examples of business visuals
- Executive KPI strip: current value, 7/30-day trend, % change, and a traffic-light indicator.
- Adoption funnel (custom ECharts funnel) combining product analytics and server-side events.
- Revenue vs. infrastructure cost stacked area chart (two data sources: billing API + cost tags from cloud infra).
We recommend exporting Business Charts visualizations as reusable templates (panel JSON) and storing them in your dashboards-as-code repository.
Best practices (engineering + product + leadership)
- Define the question first: each dashboard must answer 1–3 clear questions for its audience.
- Two-layer UX: summary KPIs at the top, drilldowns per system below.
- Consistent metrics & tags: enforce naming conventions and resource tags across teams so joins and filters work predictably.
- Dashboard performance: minimize heavy client-side joins; prefer pre-aggregated, paged data or server-side aggregation.
- Versioning & code: store dashboards as JSON / provisioning files, manage them with Git and CI (dashboards-as-code). See Grafana provisioning and API for automation. :contentReference[oaicite:4]{index=4}
Practical recipe: From vendor APIs to executive panel (step-by-step)
We follow a pragmatic 6-step recipe we use for clients:
- Inventory: list the data sources and confirm available APIs, metrics, and rates (Datadog, Dynatrace, Splunk, InfluxDB, SQL warehouses).
- Design the KPI: write a one-sentence question the KPI answers and its mathematical definition (e.g., "95th perc. latency per region over last 24h").
- Prototype: build an aggregator or try Mixed queries in a Grafana panel. Prefer a minimal working prototype over perfect modeling.
- Optimize: move heavy joins to the aggregator, add caching, ensure auth tokens are secured and rotated.
- Template & version: export the panel JSON, add it to dashboards-as-code with variables and provisioning YAML for the target Grafana instance.
- Operationalize: monitor dashboard performance (slow queries), add alerts where KPIs cross thresholds, and schedule reviews with stakeholders.
Governance, reproducibility and sharing real value
External references are useful, but we add concrete value by delivering:
- Sanitized dashboard templates: we provide cleaned JSON exports that remove company-sensitive IDs but preserve queries and visual logic.
- Dashboards-as-code: Terraform + Grafana provider or provisioning YAML to deploy dashboards reproducibly across environments.
- Hands-on workshops: live sessions where we adapt templates to your data and train product/engineering on interpretation.
- Pre-built aggregators: example code repositories (our template Node/Go Python aggregator) that demonstrate auth, retries, merging logic and caching.
Plugins & tooling we recommend
- Business Charts / Business Table / Business Suite (Volkov Labs) — for advanced business visualizations and tables. :contentReference[oaicite:5]{index=5}
- JSON API or SimpleJSON datasource — to ingest custom aggregator endpoints.
- Transformations (Grafana panel transformations) — for client-side joins and reshaping. :contentReference[oaicite:6]{index=6}
- Provisioning & dashboards-as-code — use Grafana provisioning, the HTTP API or Terraform for stable deployments. :contentReference[oaicite:7]{index=7}
Security & compliance considerations
- Ensure API keys and secrets used by aggregators are stored in secure vaults (HashiCorp Vault, AWS Secrets Manager).
- Redact or obfuscate PII and any sensitive identifiers before exposing data to dashboards used by non-technical audiences.
- Review SLAs and API rate limits — some vendor APIs charge by volume.
Examples we can share (non-proprietary)
To provide practical value without exposing customer data, we can publish:
- Sanitized Grafana dashboard JSONs for the patterns above (Mixed-Panel, API-aggregator + Business Charts sample).
- Small example aggregator (MIT / Apache-2.0) that demonstrates fetching metrics from two public sources, joining by timestamp and exposing a JSON endpoint consumable by Grafana.
- Step-by-step workshop slides and a short video walkthrough showing how to deploy and adapt the templates.
Appendix — quick configuration notes
1. Mixed datasource in Grafana
In a panel choose the Mixed datasource, then add queries where each query selects its datasource. Use the panel > Transformations to align series or use "Outer join" for tables. :contentReference[oaicite:8]{index=8}
2. Business Charts
Install from Grafana plugins catalog or by using the Grafana CLI (if self-hosted). Business Charts exposes a code editor for ECharts options and a visual editor to map data frames to the chart. Documentation and examples are on the plugin site (Volkov Labs). :contentReference[oaicite:9]{index=9}
3. Dashboards-as-code
Export your dashboard JSON (Dashboard settings → JSON model) and add provisioning YAML or use Terraform with the Grafana provider to deploy dashboards across environments.
How we can help (call-to-action)
At Observabilitrends we help teams move from “lots of graphs” to actionable dashboards that are trusted by both engineers and executives. We offer:
- Sanitised dashboard templates you can import into Grafana.
- Workshops and office-hours to adapt templates to your data.
- Implementation of aggregation services and dashboards-as-code pipelines.
Contact us at our contact page or email info@observabilitrends.com for a free scoping call.
References: Grafana docs on datasources & transformations; Grafana best-practices; Volkov Labs Business Charts & Business Suite docs and plugin pages. :contentReference[oaicite:10]{index=10}