Interview Questions for a Site Reliability Engineer

How do you define a meaningful SLO and SLI for a critical service, and how does the error budget change team behavior?

What to look for: Picks user-facing SLIs (availability, latency percentiles), sets a realistic SLO, and explains how a burned error budget gates releases or shifts focus to reliability work.

Walk me through how you'd add observability to a service that currently only has basic logs.

What to look for: Adds RED/USE-style metrics, structured logs, and distributed tracing with tools like Prometheus, Grafana, and OpenTelemetry, and defines actionable alerts tied to SLOs, not noise.

Describe how you lead a production incident from page to resolution.

What to look for: Establishes incident command, focuses on mitigation before root cause, communicates status clearly, and follows with a blameless post-mortem and tracked remediation items.

How do you identify and eliminate toil, and how do you decide what to automate first?

What to look for: Quantifies toil as repetitive, manual, automatable work, prioritizes by frequency and risk, and builds tooling, runbooks, or self-healing systems with measurable reduction.

Walk me through designing a progressive delivery setup with canary deployments and automated rollback.

What to look for: Describes canary or blue-green rollout, health and SLO-based promotion gates, automated rollback triggers, and metrics that decide whether to proceed.

You run production workloads on Kubernetes. How do you keep them reliable and right-sized at scale?

What to look for: Covers resource requests/limits, autoscaling, pod disruption budgets, readiness/liveness probes, node capacity, and avoiding noisy-neighbor and cascading-failure issues.

Tell me about the worst incident you've handled. How did you respond and what did the post-mortem change?

What to look for: Calm, structured response under pressure, honest blameless analysis, and concrete systemic fixes rather than blaming a person or stopping at a hotfix.

Describe a time you improved a system's reliability with a measurable result.

What to look for: Specific reliability problem, the change made (redundancy, automation, better alerting), and a metric like reduced incidents, MTTR, or improved availability.

Tell me about a time you reduced significant on-call toil or alert fatigue.

What to look for: Identifies noisy or manual sources, tunes alerts to be actionable, automates remediation, and shows the on-call experience genuinely improved.

Describe a disagreement with a product team over shipping versus reliability. How did you resolve it?

What to look for: Uses error budgets and data to frame the trade-off, finds a path that respects both velocity and risk, and avoids being either a blocker or a pushover.

Give an example of a capacity or performance problem you caught before it caused an outage.

What to look for: Proactive capacity planning or load testing, spotting a trend early, and intervening before users were affected.

A service is burning its error budget fast mid-quarter. What do you do?

What to look for: Pauses risky releases, investigates the dominant failure mode, prioritizes reliability work, and communicates the trade-off with product owners using the budget as the lever.

Latency is spiking intermittently in production but you can't reproduce it. How do you investigate?

What to look for: Uses traces and percentile metrics to localize, correlates with deploys/traffic/dependencies, looks for tail latency causes like GC, contention, or a slow downstream, and forms testable hypotheses.

How would you design and run a chaos experiment to validate that a service survives a dependency failure?

What to look for: Defines a steady-state hypothesis, limits blast radius, injects a controlled failure (node, network, dependency), and verifies graceful degradation and alerting fire correctly.

You're expecting a 5x traffic spike for a launch. How do you prepare?

What to look for: Load tests to find limits, plans autoscaling and capacity headroom, checks dependencies and rate limits, prepares rollback and feature flags, and sets up monitoring and an incident plan.

A team wants to ship a feature with no monitoring or rollback plan. How do you handle it?

What to look for: Treats reliability as a design-time gate, partners on adding SLO-backed monitoring and rollback, and frames it as enabling speed safely rather than as a blocker.

How do you make reliability a shared responsibility with development teams rather than your team's burden?

What to look for: Embeds SLOs, observability, and runbooks into the development lifecycle, shares on-call or production ownership, and builds a blameless, learning culture.

How do you run a blameless post-mortem so people are honest and real fixes happen?

What to look for: Focuses on systems and contributing factors over individuals, drives clear action items with owners, and follows up to ensure they're completed.

How do you communicate during a high-severity incident to both engineers and stakeholders?

What to look for: Provides clear, regular status with impact and ETA, separates the technical channel from stakeholder updates, and avoids speculation while keeping people informed.

How do you push back on an unrealistic reliability expectation without becoming the team that says no?

What to look for: Uses data and error budgets to set realistic targets, offers options and trade-offs, and partners on a path that balances cost, risk, and velocity.