Right, so you’ve got Prometheus scraping all those lovely metrics. Congratulations, you now have a firehose of data pointed directly at your face. Grafana is how you put a nozzle on that hose and actually see what’s going on. It’s the difference between staring at a spreadsheet of numbers and looking at a beautifully rendered graph that tells you, “Hey, your service is on fire.” Let’s get you from data to dashboard.

The Quick Win: Importing a Dashboard

The Grafana community is your brilliant, over-caffeinated friend who has already built a dashboard for probably every piece of software ever written. Before you spend hours building something from scratch, see if someone has already done the heavy lifting for you.

Head over to grafana.com/grafana/dashboards and search for what you need, say, “Node Exporter Full.” You’ll find a dashboard with an ID, like 1860. In your Grafana UI, you navigate to Dashboards > New > Import and paste that ID. Hit Load, and you’re presented with a configuration screen. This is the most important part: you must tell Grafana where to find the data for this dashboard by selecting your Prometheus data source from the dropdown. If you forget this, you’ll get a beautiful dashboard full of “No data” panels, which is about as useful as a chocolate teapot.

Once you select the data source and hit Import, boom. You have a fully functional, professional-grade dashboard. It’s a fantastic way to get immediate value and to see how experienced dashboard authors structure their queries and visualizations. It’s like getting a free apprenticeship.

Building from Scratch: Your First Real Dashboard

But imported dashboards are a starting point. To truly own your observability, you need to build your own. Click the big + icon in the sidebar and select Dashboard. You’ll be greeted by a blank, slightly intimidating canvas. Click Add visualization to add your first panel.

This is where the magic happens. The Panel Editor is your command center. You’ll live in the Query tab. Here, you must select your data source (again, Prometheus) and then you start building your PromQL queries.

Let’s say you want to track the average request latency for a service. You might start with a query like this:

rate(http_request_duration_seconds_sum{job="my-api"}[5m]) / rate(http_request_duration_seconds_count{job="my-api"}[5m])

This is a classic PromQL pattern for calculating a average latency from a Summary metric. Now, why the [5m] range and why rate? This is the critical part. rate() automatically handles counter resets and gives you a per-second average over that window, which is almost always what you want for a graph. The [5m] is a smoothing factor; a longer window gives you a smoother, less jittery line, but reacts more slowly to changes. A 1m window will show you every little spike. Choose based on whether you’re diagnosing a sudden outage or watching long-term trends.

The Art of the Graph (and Why Pie Charts Are Evil)

Grafana will render your query with default settings, which are often… bad. This is where you move from data to information. Click on the Visualization tab in the Panel Editor.

  • Right Y-axis, left Y-axis: Is your value in milliseconds or seconds? Does it max out at 0.1 or 1000? Click Axis in the visualization settings and set a Unit (e.g., time -> milliseconds). This instantly makes your graph readable by humans.
  • Standard options overrides: This is Grafana’s killer feature. You can change how a specific series looks. Is that job="my-api" line spiking? Override its color to bright red. Is the path="/health" series meaningless noise? Add an override to Exclude it from the legend or even hide it entirely. This is how you declutter.
  • Thresholds: Add a threshold line at, say, 500ms. Now you can instantly see when you’re breaching your SLO. You can even set the background to change color (yellow for warning, red for critical) based on these values.

And a word of advice from the trenches: just say no to pie charts for metrics. They are utterly useless for understanding time-series data. A line graph showing error rates over time is insightful; a pie chart showing “errors vs. successes” at one arbitrary moment in time is not.

Variables: Making Your Dashboards Dynamic

A static dashboard is fine for a single service. A dashboard that can show you data for any service, any pod, or any environment? That’s a superpower. This is where Template Variables come in.

Go to your dashboard settings (the gear icon) and select Variables. Let’s add a variable to select a job.

  1. Name: job
  2. Type: Query
  3. Data source: Your Prometheus source
  4. Query: label_values(job) (this queries Prometheus for all values of the job label)

Now, go back to your panel and edit your query. You can now use this variable to make it dynamic:

rate(http_request_duration_seconds_sum{job=~"$job"}[5m]) / rate(http_request_duration_seconds_count{job=~"$job"}[5m])

Note the use of =~ (the regex match operator) instead of = because the variable might allow multi-select. Now, at the top of your dashboard, you’ll get a handy dropdown where you (or your on-call engineer at 3 AM) can select which service to look at. You can create variables for instance, pod name, environment—anything that’s a label in your metrics. It transforms your dashboard from a single-purpose tool into a central observability console.

The final, non-negotiable step? Save your dashboard. Give it a clear name and a useful folder. An unsaved dashboard is a fleeting dream. And for the love of all that is holy, add a descriptive description. Your future self, wondering what the “Temporary Dashboard (copy 3)” is for, will thank you.