Home › Blog › Prometheus Interview Questions (2026): 30 Questions on PromQL, Alerting and Scaling

Prometheus Interview Questions (2026): 30 Questions on PromQL, Alerting and Scaling

Prometheus interview questions test five things: the pull-based scrape model, the four metric types, PromQL (above all rate() , sum by and histogram_quantile() ), alerting through Alertmanager, and keeping cardinality and storage under control. Junior candidates are asked to explain the data model; senior candidates are asked to debug a missing alert or a memory spike and to design long-term storage.

Prometheus interview questions test five things: the pull-based scrape model, the four metric types, PromQL (above all rate(), sum by and histogram_quantile()), alerting through Alertmanager, and keeping cardinality and storage under control. Junior candidates are asked to explain the data model; senior candidates are asked to debug a missing alert or a memory spike and to design long-term storage.

Below are 30 questions grouped by level with concise answers, three troubleshooting scenarios, and how interviews usually test the tool. Defaults quoted here (1-minute scrape and evaluation intervals, 10-second scrape timeout, 15-day local retention, 5-minute lookback) are as the Prometheus documentation listed them on November 17, 2026. For the wider role, see SRE interview questions.

Prometheus fundamentals: questions 1–9

  1. What is Prometheus, and what is it not? An open-source monitoring system and time series database that scrapes numeric metrics over HTTP, stores them locally and evaluates recording and alerting rules. It is not a log store, a tracing system or a clustered long-term database on its own.
  2. Why does Prometheus pull instead of push? The server scrapes each target's /metrics endpoint on a schedule, so it knows when a target is down (the up series drops to 0), controls its own load, and you can curl an endpoint to see exactly what is exposed. Short-lived batch jobs can push to the Pushgateway instead.
  3. Name the four metric types. Counter (only increases, resets on restart, e.g. http_requests_total), gauge (goes up and down, e.g. memory in use), histogram (bucketed counts plus _sum and _count) and summary (quantiles calculated in the client, plus sum and count).
  4. What makes a time series unique? The metric name plus its full set of labels. http_requests_total{method="GET",status="500"} and the same metric with status="200" are two series, which is why labels drive memory use.
  5. What is an exporter? A process that translates another system's statistics into the Prometheus format, such as node_exporter for Linux hosts or the blackbox exporter for HTTP and TCP probes. Applications you own should expose metrics directly through a client library.
  6. Instant vector vs range vector? An instant vector has one sample per series at a point in time (up); a range vector has samples over a window (up[5m]). Functions such as rate() take range vectors, and you can't graph a range vector directly.
  7. What does rate() do? It returns the per-second average increase of a counter over the window and handles counter resets: rate(http_requests_total[5m]) is requests per second. Don't apply it to gauges; the docs warn the result is nonsensical. Use deriv() or delta() there.
  8. What are the default scrape and evaluation intervals? The global scrape_interval and evaluation_interval both default to 1 minute and scrape_timeout to 10 seconds. Most teams set 15 to 30 seconds explicitly.
  9. What does Alertmanager do? Prometheus decides when an alert condition is true; Alertmanager receives the firing alerts, deduplicates, groups, silences and inhibits them, and routes them to receivers such as email, chat or a paging tool.

Intermediate Prometheus questions: 10–18

  1. rate() vs irate() vs increase()? rate() averages over the whole window and suits alerts and dashboards; irate() uses only the last two samples, so it is spiky and fits fast-moving graphs; increase() is the total rise over the window, extrapolated, so it can be a non-integer.
  2. How do you get p95 latency across all instances? Aggregate histogram buckets before taking the quantile: histogram_quantile(0.95, sum by (le) (rate(http_request_duration_seconds_bucket[5m]))). Averaging summary quantiles across instances is wrong, and the docs say so.
  3. Histogram vs summary? Histograms can be aggregated and their quantiles are computed on the server, at the cost of choosing buckets. Summaries are precise per instance but can't be aggregated. The current docs recommend native histograms over both where you can use them.
  4. What are recording rules for? Precomputing an expensive expression on a schedule and saving it as a new series, named by the level:metric:operations convention such as job:http_requests:rate5m. Dashboards get faster and alert expressions stay simple.
  5. Walk through an alerting rule, including for. expr is the PromQL condition; for: 10m keeps the alert pending until the condition has held for 10 minutes, which filters flapping; labels carry severity for routing; annotations carry the summary and runbook link.
  6. What is relabeling? relabel_configs rewrite target labels before the scrape (keep, drop, rename, build the address from service-discovery metadata); metric_relabel_configs act on scraped samples before storage, for example to drop a high-cardinality metric.
  7. How does discovery work on Kubernetes? kubernetes_sd_configs lists pods, services, endpoints or nodes from the API server and relabeling keeps the targets you want. Many clusters use the Prometheus Operator with ServiceMonitor and PodMonitor resources instead of raw scrape config.
  8. What is cardinality and why does it matter? The number of unique series. Labels with unbounded values such as user IDs, request IDs or full URLs multiply series, raising memory, slowing queries and eventually crashing the server. Find offenders on the TSDB status page or with promtool tsdb analyze.
  9. How is data stored locally? Samples go into an in-memory head block protected by a write-ahead log, are cut into two-hour blocks on disk and compacted into larger ones. Retention defaults to 15 days unless you set --storage.tsdb.retention.time or a size limit.

Experienced Prometheus questions: 19–27

  1. How do you scale Prometheus? Vertically first, then functional sharding (one server per team, cluster or region), hashmod sharding of targets for very large jobs, and federation or remote write for a global view. For long-term, highly available storage, teams remote-write to a compatible backend such as Thanos, Cortex or Mimir.
  2. How do you make it highly available? Run two identical replicas scraping the same targets and send both to a clustered Alertmanager, which deduplicates notifications. Prometheus doesn't replicate data itself, so query-side deduplication comes from a layer such as Thanos Query.
  3. What is remote write and what goes wrong with it? Prometheus streams samples from its WAL to a remote endpoint. Common failures: the queue falls behind (watch the prometheus_remote_storage_* metrics), the receiver throttles, or you ship far more series than planned. Trim with write_relabel_configs.
  4. What is agent mode? A mode started with the --agent flag that scrapes and forwards through remote write but does no local querying, rule evaluation or long-term storage. It suits edge sites and many small clusters.
  5. What changed in Prometheus 3? The migration guide lists UTF-8 metric and label names, left-open range selectors (so a subquery like [1m:1m] can return one point fewer), holt_winters renamed to double_exponential_smoothing behind a feature flag, Alertmanager API v1 removed, and stricter scrape Content-Type handling.
  6. How do you alert on an SLO? Use multi-window, multi-burn-rate alerts on an error ratio, as described in Google's SRE Workbook: for a 99.9% target over 30 days, page when both the 1-hour and 5-minute burn rates exceed 14.4, and open a ticket for slower burns. Recording rules hold the ratios.
  7. How do you catch a target that vanished entirely? up == 0 fires only while the target is still discovered. If it drops out of service discovery, use absent(up{job="api"}), or alert on the expected count with count(up{job="api"}) < 3.
  8. What is staleness? When a series stops being scraped, Prometheus writes a staleness marker so queries stop returning it rather than repeating the last value. Without a marker, an instant query looks back up to 5 minutes by default (--query.lookback-delta).
  9. How do grouping, inhibition and silences cut alert noise? group_by with group_wait, group_interval and repeat_interval bundles related alerts into one notification; inhibition mutes lower-severity alerts while a related critical one fires; silences mute alerts during planned work.

Prometheus scenario and troubleshooting questions: 28–30

  1. An alert should have fired during an outage but didn't. Where do you look? In order: is the rule loaded (Rules page, promtool check rules); does the expression return data in the graph view for that time; was for longer than the incident; was the series stale or absent; did Prometheus send it (Alerts page, prometheus_notifications_* metrics); did Alertmanager group, inhibit or silence it; did the receiver fail? Fix the gap and add a promtool test rules case.
  2. Prometheus memory doubled after an application deploy. Suspect new high-cardinality labels. Compare prometheus_tsdb_head_series before and after, find the metric on the TSDB status page, and drop or rewrite the label with metric_relabel_configs while the team fixes its instrumentation. Set sample_limit on the job so it can't happen silently again.
  3. A request-rate graph drops to zero every few hours. Check up for the target flapping, scrape duration against scrape_timeout, and whether the range is too short for the scrape interval; a common rule of thumb is a window of at least four scrape intervals. Then check resets() for restarts.

How interviews test Prometheus

RoundWhat they askWhat a strong answer shows
Recruiter or phone screenPull model, metric types, exporters, what Alertmanager doesShort, correct definitions with an example metric
Technical roundWrite PromQL: error ratio, p95 latency, top five endpoints by trafficCorrect use of rate(), sum by and histogram_quantile()
On-call or scenario roundMissing alert, memory growth, slow dashboardsAn ordered debugging path instead of guesses
Senior or design roundHA, long-term storage, many clustersTrade-offs between federation, sharding and remote write

Candidates for SRE and platform roles commonly report being asked to write two or three queries by hand and to explain a past alert they tuned. The same ground appears in DevOps interview questions and Kubernetes interview questions, and the design side is in the metrics monitoring system walkthrough. Practise by running Prometheus locally against node_exporter and writing ten queries out loud.

Where CoPilot Interview fits

CoPilot Interview is an AI interview assistant whose free AI mock interview builds Prometheus and monitoring questions from your résumé and the job description, reads them aloud and scores each spoken answer out of 10, which is a quick way to find the answers above that you can't yet explain without notes. The free plan gives 15 minutes a day of live help in sessions of up to 15 minutes, with AI mock interviews included. The browser demo at /try has 145 practice questions in 10 tracks with typed answers scored out of 10, no signup needed. Paid plans are Standard at $24.99 a month and Pro at $39.99 a month, and the desktop app runs on Windows 10 and 11 and macOS 12 or later. Technical screens, take-home tasks, online assessments and proctored tests usually forbid outside help, so prepare for those rather than use any tool during them; in any live interview, follow your employer's rules.

Disclosure: CoPilot Interview is our product, so weigh what we say about it accordingly.

Frequently asked questions

What PromQL queries should I know by heart?

An error ratio such as <code>sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))</code>, a p95 from histogram buckets, <code>topk(5, ...)</code>, <code>up == 0</code>, <code>absent()</code>, and a disk-fill prediction with <code>predict_linear(node_filesystem_avail_bytes[6h], 4*3600) &lt; 0</code>.

What is the difference between Prometheus and Grafana?

Prometheus collects, stores and alerts on metrics; Grafana visualises data from Prometheus and many other sources. Interviews often treat them as a pair, but they are separate projects.

What is the default data retention in Prometheus?

15 days of local storage unless you set a retention time or size, according to the Prometheus storage documentation as of November 17, 2026. Longer history usually goes to a remote-write backend.

How long does it take to prepare for a Prometheus interview?

If you already run services, a week of evenings covers the data model, PromQL and Alertmanager. Spend most of it writing queries against a real setup and explaining one alert you would design for a service you know.

Practice this interview before the real one

The AI mock interview builds questions from your resume and the job description, reads them aloud and scores each spoken answer out of 10. Included on the free plan.

Start a mock interview