Prometheus interview questions test five things: the pull-based scrape model, the four metric types, PromQL (above all rate(), sum by and histogram_quantile()), alerting through Alertmanager, and keeping cardinality and storage under control. Junior candidates are asked to explain the data model; senior candidates are asked to debug a missing alert or a memory spike and to design long-term storage.
Below are 30 questions grouped by level with concise answers, three troubleshooting scenarios, and how interviews usually test the tool. Defaults quoted here (1-minute scrape and evaluation intervals, 10-second scrape timeout, 15-day local retention, 5-minute lookback) are as the Prometheus documentation listed them on November 17, 2026. For the wider role, see SRE interview questions.
Prometheus fundamentals: questions 1–9
- What is Prometheus, and what is it not? An open-source monitoring system and time series database that scrapes numeric metrics over HTTP, stores them locally and evaluates recording and alerting rules. It is not a log store, a tracing system or a clustered long-term database on its own.
- Why does Prometheus pull instead of push? The server scrapes each target's
/metricsendpoint on a schedule, so it knows when a target is down (theupseries drops to 0), controls its own load, and you can curl an endpoint to see exactly what is exposed. Short-lived batch jobs can push to the Pushgateway instead. - Name the four metric types. Counter (only increases, resets on restart, e.g.
http_requests_total), gauge (goes up and down, e.g. memory in use), histogram (bucketed counts plus_sumand_count) and summary (quantiles calculated in the client, plus sum and count). - What makes a time series unique? The metric name plus its full set of labels.
http_requests_total{method="GET",status="500"}and the same metric withstatus="200"are two series, which is why labels drive memory use. - What is an exporter? A process that translates another system's statistics into the Prometheus format, such as node_exporter for Linux hosts or the blackbox exporter for HTTP and TCP probes. Applications you own should expose metrics directly through a client library.
- Instant vector vs range vector? An instant vector has one sample per series at a point in time (
up); a range vector has samples over a window (up[5m]). Functions such asrate()take range vectors, and you can't graph a range vector directly. - What does
rate()do? It returns the per-second average increase of a counter over the window and handles counter resets:rate(http_requests_total[5m])is requests per second. Don't apply it to gauges; the docs warn the result is nonsensical. Usederiv()ordelta()there. - What are the default scrape and evaluation intervals? The global
scrape_intervalandevaluation_intervalboth default to 1 minute andscrape_timeoutto 10 seconds. Most teams set 15 to 30 seconds explicitly. - What does Alertmanager do? Prometheus decides when an alert condition is true; Alertmanager receives the firing alerts, deduplicates, groups, silences and inhibits them, and routes them to receivers such as email, chat or a paging tool.
Intermediate Prometheus questions: 10–18
- rate() vs irate() vs increase()?
rate()averages over the whole window and suits alerts and dashboards;irate()uses only the last two samples, so it is spiky and fits fast-moving graphs;increase()is the total rise over the window, extrapolated, so it can be a non-integer. - How do you get p95 latency across all instances? Aggregate histogram buckets before taking the quantile:
histogram_quantile(0.95, sum by (le) (rate(http_request_duration_seconds_bucket[5m]))). Averaging summary quantiles across instances is wrong, and the docs say so. - Histogram vs summary? Histograms can be aggregated and their quantiles are computed on the server, at the cost of choosing buckets. Summaries are precise per instance but can't be aggregated. The current docs recommend native histograms over both where you can use them.
- What are recording rules for? Precomputing an expensive expression on a schedule and saving it as a new series, named by the
level:metric:operationsconvention such asjob:http_requests:rate5m. Dashboards get faster and alert expressions stay simple. - Walk through an alerting rule, including
for.expris the PromQL condition;for: 10mkeeps the alert pending until the condition has held for 10 minutes, which filters flapping;labelscarry severity for routing;annotationscarry the summary and runbook link. - What is relabeling?
relabel_configsrewrite target labels before the scrape (keep, drop, rename, build the address from service-discovery metadata);metric_relabel_configsact on scraped samples before storage, for example to drop a high-cardinality metric. - How does discovery work on Kubernetes?
kubernetes_sd_configslists pods, services, endpoints or nodes from the API server and relabeling keeps the targets you want. Many clusters use the Prometheus Operator with ServiceMonitor and PodMonitor resources instead of raw scrape config. - What is cardinality and why does it matter? The number of unique series. Labels with unbounded values such as user IDs, request IDs or full URLs multiply series, raising memory, slowing queries and eventually crashing the server. Find offenders on the TSDB status page or with
promtool tsdb analyze. - How is data stored locally? Samples go into an in-memory head block protected by a write-ahead log, are cut into two-hour blocks on disk and compacted into larger ones. Retention defaults to 15 days unless you set
--storage.tsdb.retention.timeor a size limit.
Experienced Prometheus questions: 19–27
- How do you scale Prometheus? Vertically first, then functional sharding (one server per team, cluster or region), hashmod sharding of targets for very large jobs, and federation or remote write for a global view. For long-term, highly available storage, teams remote-write to a compatible backend such as Thanos, Cortex or Mimir.
- How do you make it highly available? Run two identical replicas scraping the same targets and send both to a clustered Alertmanager, which deduplicates notifications. Prometheus doesn't replicate data itself, so query-side deduplication comes from a layer such as Thanos Query.
- What is remote write and what goes wrong with it? Prometheus streams samples from its WAL to a remote endpoint. Common failures: the queue falls behind (watch the
prometheus_remote_storage_*metrics), the receiver throttles, or you ship far more series than planned. Trim withwrite_relabel_configs. - What is agent mode? A mode started with the
--agentflag that scrapes and forwards through remote write but does no local querying, rule evaluation or long-term storage. It suits edge sites and many small clusters. - What changed in Prometheus 3? The migration guide lists UTF-8 metric and label names, left-open range selectors (so a subquery like
[1m:1m]can return one point fewer),holt_wintersrenamed todouble_exponential_smoothingbehind a feature flag, Alertmanager API v1 removed, and stricter scrape Content-Type handling. - How do you alert on an SLO? Use multi-window, multi-burn-rate alerts on an error ratio, as described in Google's SRE Workbook: for a 99.9% target over 30 days, page when both the 1-hour and 5-minute burn rates exceed 14.4, and open a ticket for slower burns. Recording rules hold the ratios.
- How do you catch a target that vanished entirely?
up == 0fires only while the target is still discovered. If it drops out of service discovery, useabsent(up{job="api"}), or alert on the expected count withcount(up{job="api"}) < 3. - What is staleness? When a series stops being scraped, Prometheus writes a staleness marker so queries stop returning it rather than repeating the last value. Without a marker, an instant query looks back up to 5 minutes by default (
--query.lookback-delta). - How do grouping, inhibition and silences cut alert noise?
group_bywithgroup_wait,group_intervalandrepeat_intervalbundles related alerts into one notification; inhibition mutes lower-severity alerts while a related critical one fires; silences mute alerts during planned work.
Prometheus scenario and troubleshooting questions: 28–30
- An alert should have fired during an outage but didn't. Where do you look? In order: is the rule loaded (Rules page,
promtool check rules); does the expression return data in the graph view for that time; wasforlonger than the incident; was the series stale or absent; did Prometheus send it (Alerts page,prometheus_notifications_*metrics); did Alertmanager group, inhibit or silence it; did the receiver fail? Fix the gap and add apromtool test rulescase. - Prometheus memory doubled after an application deploy. Suspect new high-cardinality labels. Compare
prometheus_tsdb_head_seriesbefore and after, find the metric on the TSDB status page, and drop or rewrite the label withmetric_relabel_configswhile the team fixes its instrumentation. Setsample_limiton the job so it can't happen silently again. - A request-rate graph drops to zero every few hours. Check
upfor the target flapping, scrape duration againstscrape_timeout, and whether the range is too short for the scrape interval; a common rule of thumb is a window of at least four scrape intervals. Then checkresets()for restarts.
How interviews test Prometheus
| Round | What they ask | What a strong answer shows |
|---|---|---|
| Recruiter or phone screen | Pull model, metric types, exporters, what Alertmanager does | Short, correct definitions with an example metric |
| Technical round | Write PromQL: error ratio, p95 latency, top five endpoints by traffic | Correct use of rate(), sum by and histogram_quantile() |
| On-call or scenario round | Missing alert, memory growth, slow dashboards | An ordered debugging path instead of guesses |
| Senior or design round | HA, long-term storage, many clusters | Trade-offs between federation, sharding and remote write |
Candidates for SRE and platform roles commonly report being asked to write two or three queries by hand and to explain a past alert they tuned. The same ground appears in DevOps interview questions and Kubernetes interview questions, and the design side is in the metrics monitoring system walkthrough. Practise by running Prometheus locally against node_exporter and writing ten queries out loud.
Where CoPilot Interview fits
CoPilot Interview is an AI interview assistant whose free AI mock interview builds Prometheus and monitoring questions from your résumé and the job description, reads them aloud and scores each spoken answer out of 10, which is a quick way to find the answers above that you can't yet explain without notes. The free plan gives 15 minutes a day of live help in sessions of up to 15 minutes, with AI mock interviews included. The browser demo at /try has 145 practice questions in 10 tracks with typed answers scored out of 10, no signup needed. Paid plans are Standard at $24.99 a month and Pro at $39.99 a month, and the desktop app runs on Windows 10 and 11 and macOS 12 or later. Technical screens, take-home tasks, online assessments and proctored tests usually forbid outside help, so prepare for those rather than use any tool during them; in any live interview, follow your employer's rules.
Disclosure: CoPilot Interview is our product, so weigh what we say about it accordingly.
Frequently asked questions
What PromQL queries should I know by heart?
An error ratio such as <code>sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))</code>, a p95 from histogram buckets, <code>topk(5, ...)</code>, <code>up == 0</code>, <code>absent()</code>, and a disk-fill prediction with <code>predict_linear(node_filesystem_avail_bytes[6h], 4*3600) < 0</code>.
What is the difference between Prometheus and Grafana?
Prometheus collects, stores and alerts on metrics; Grafana visualises data from Prometheus and many other sources. Interviews often treat them as a pair, but they are separate projects.
What is the default data retention in Prometheus?
15 days of local storage unless you set a retention time or size, according to the Prometheus storage documentation as of November 17, 2026. Longer history usually goes to a remote-write backend.
How long does it take to prepare for a Prometheus interview?
If you already run services, a week of evenings covers the data model, PromQL and Alertmanager. Spend most of it writing queries against a real setup and explaining one alert you would design for a service you know.
Practice this interview before the real one
The AI mock interview builds questions from your resume and the job description, reads them aloud and scores each spoken answer out of 10. Included on the free plan.
Start a mock interview