Skip to content
SkillPilotNavigate Your Career

Database Reliability Engineer Roadmap

A path into database reliability engineering — replication and consistency, restores you have actually verified, zero-downtime schema migrations, corruption detection, databases on Kubernetes, and RTO and RPO as a contract.

Entry level
$129,281
Database Reliability Engineer total pay estimate, 25th percentile (US, 24 salaries submitted)
Mid level
$156,093
Database Reliability Engineer total pay estimate (US average, 24 salaries submitted)
Senior level
$190,972
Database Reliability Engineer total pay estimate, 75th percentile (US, 24 salaries submitted)
Glassdoor

Search for this role and you mostly find job listings, not a path into it — the page candidates are looking for barely exists. This roadmap treats the database as a production system with an owner rather than as a dependency someone else maintains: restores that are verified on a schedule, migrations that ship without downtime, and corruption caught by a check rather than by a customer.

New to Linux and the command line?

This path assumes fundamentals you may not have yet. Our Foundations Pack is out and free — Linux, the shell and Git, with exercises that mark your work and explain why you got it wrong. We're writing an agents pack next; leave your email if you want to hear when it ships.

One email when the pack launches. No spam, unsubscribe any time.

Your progress0%

The path, phase by phase

  1. Database Fundamentals for Operators

    Operating a database means understanding what happens below the query: how pages reach disk, why the write-ahead log exists, and what a transaction actually guarantees. Without that model every incident is a mystery. Done when you can trace a single INSERT from client connection through WAL to durable storage, and explain which step is lost if the process is killed at each point.

    4-5 weeks
    7 Skills
    Storage Engines and Page Layout
    Write-Ahead Logging
    ACID and Isolation Levels
    MVCC and Vacuum
    Index Internals
    Query Planning
    Connection Handling
    Show details, projects and resources

    Skills you'll master

    Storage Engines and Page Layoutintermediate
    Write-Ahead Loggingadvanced
    ACID and Isolation Levelsadvanced
    MVCC and Vacuumadvanced
    Index Internalsadvanced
    Query Planningintermediate
    Connection Handlingintermediate

    Hands-on projects

    1. 01Trace one INSERT from connection to durable storage and record what is lost if the process dies at each stage
    2. 02Reproduce a phantom read and a non-repeatable read on purpose, then fix each by changing only the isolation level
    3. 03Force a table bloat scenario, observe what vacuum reclaims and what it does not, and record the disk numbers
    4. 04Take a slow query, read its plan, and make it fast by changing the index rather than the SQL
    5. 05Exhaust a connection pool deliberately and record what the application error looks like from the user's side

  2. Replication and Consistency

    Replication is how databases survive hardware failure and also how they silently serve stale data. The distinction between synchronous and asynchronous is not a configuration detail: it decides whether a failover can lose committed transactions. Done when you have measured replication lag under write load and can state exactly how much data your topology loses if the primary dies right now.

    4-6 weeks
    7 Skills
    Synchronous vs Asynchronous Replication
    Streaming and Logical Replication
    Replication Lag Measurement
    Read Replica Routing
    Quorum and Consensus
    Split-brain Prevention
    Cross-region Replication
    Show details, projects and resources

    Skills you'll master

    Synchronous vs Asynchronous Replicationadvanced
    Streaming and Logical Replicationadvanced
    Replication Lag Measurementadvanced
    Read Replica Routingintermediate
    Quorum and Consensusadvanced
    Split-brain Preventionadvanced
    Cross-region Replicationadvanced

    Hands-on projects

    1. 01Build a primary with two replicas, then measure lag under sustained write load rather than at idle
    2. 02Calculate your topology's real data loss window on primary failure, and verify it by killing the primary mid-write
    3. 03Route reads to a replica and find a user-visible bug caused by reading your own stale write
    4. 04Set up logical replication between two different major versions and record which DDL changes it refuses to carry
    5. 05Induce a network partition between primary and replica and document what each side believes about the other
    6. 06Compare synchronous replication's latency cost against its durability gain, with numbers from your own hardware

  3. Backup and Restore Verification

    A backup that has never been restored is a hypothesis, and the moment you discover it was wrong is the worst possible moment. This phase treats the restore, not the backup, as the deliverable. Done when a restore runs automatically on a schedule, its duration is recorded as a metric, and someone is alerted when a restore fails rather than when a backup does.

    3-4 weeks
    6 Skills
    Physical and Logical Backups
    Point-in-Time Recovery
    Automated Restore Testing
    Backup Retention Design
    Restore Time Measurement
    Backup Encryption and Access
    Show details, projects and resources

    Skills you'll master

    Physical and Logical Backupsintermediate
    Point-in-Time Recoveryadvanced
    Automated Restore Testingadvanced
    Backup Retention Designintermediate
    Restore Time Measurementadvanced
    Backup Encryption and Accessintermediate

    Hands-on projects

    1. 01Restore a production-sized backup into a clean environment and publish the real restore time next to the promised one
    2. 02Perform a point-in-time recovery to a timestamp thirty seconds before a deliberate destructive statement
    3. 03Automate a nightly restore into a throwaway environment, with an alert that fires on restore failure rather than backup failure
    4. 04Design a retention policy against a real regulatory or business requirement and cost it at your current data growth
    5. 05Corrupt a backup file on purpose and confirm your verification step detects it before you need the backup

  4. High Availability and Failover

    Automatic failover is a promise that the system will make a correct decision during the worst five minutes of the quarter, and untested automation usually makes it worse. Done when you have triggered a failover in anger — killing the primary without warning the tooling — and the cluster converged on a single writer with the data loss you predicted, not more.

    4-5 weeks
    7 Skills
    Failover Automation
    Leader Election and Fencing
    Connection Pooling and Proxies
    Health Checking Design
    Switchover vs Failover
    Client Reconnection Behaviour
    Failover Testing in Production
    Show details, projects and resources

    Skills you'll master

    Failover Automationadvanced
    Leader Election and Fencingadvanced
    Connection Pooling and Proxiesadvanced
    Health Checking Designadvanced
    Switchover vs Failoverintermediate
    Client Reconnection Behaviouradvanced
    Failover Testing in Productionadvanced

    Hands-on projects

    1. 01Kill a primary without warning the failover tooling and record the time to a single healthy writer
    2. 02Configure fencing and prove that the demoted primary cannot accept a write after being isolated
    3. 03Put a connection pooler in front of a cluster and measure how many client errors a failover produces with and without it
    4. 04Write a health check that distinguishes a slow database from a dead one, and test it against both conditions
    5. 05Perform a planned switchover during working hours with zero failed writes, and document the sequence that made it safe
    6. 06Test what your application does when the database is read-only for ninety seconds

  5. Performance and Query Optimisation

    Most database performance work is not tuning the server: it is finding the three queries responsible for most of the load and fixing them. The skill is measurement discipline, because intuition about what is slow is reliably wrong. Done when you have found the top queries by total time rather than by worst case, and made a measured improvement to the one that mattered.

    4-5 weeks
    7 Skills
    Query Plan Analysis
    Index Design and Maintenance
    Workload Profiling
    Lock Contention Diagnosis
    Configuration Tuning
    Partitioning Strategy
    Cache Hit Ratio Analysis
    Show details, projects and resources

    Skills you'll master

    Query Plan Analysisadvanced
    Index Design and Maintenanceadvanced
    Workload Profilingadvanced
    Lock Contention Diagnosisadvanced
    Configuration Tuningintermediate
    Partitioning Strategyadvanced
    Cache Hit Ratio Analysisintermediate

    Hands-on projects

    1. 01Rank queries by total accumulated time rather than worst single execution, and fix the one at the top
    2. 02Find an unused index in a production-like database, prove it is unused, and measure the write throughput gained by dropping it
    3. 03Reproduce a lock contention incident, identify the blocking chain, and record how you would have found it under pressure
    4. 04Partition a large table and measure both the query improvement and the new maintenance burden it created
    5. 05Change one memory-related configuration parameter and prove the effect with before-and-after measurements
    6. 06Build a query performance dashboard that would let an on-call engineer identify a regression in under two minutes

  6. Schema Migrations Without Downtime

    A schema change is a deployment that can lock a table and take the site down, and the naive version of the migration is usually the dangerous one. Expand-and-contract turns one risky change into several safe ones. Done when you have shipped a column rename across a running system with no downtime, using separate deployments to add, backfill, switch and remove.

    3-4 weeks
    6 Skills
    Expand and Contract Pattern
    Online Schema Change Tools
    Backfill Strategy
    Lock Analysis for DDL
    Migration Rollback Design
    Application and Schema Version Skew
    Show details, projects and resources

    Skills you'll master

    Expand and Contract Patternadvanced
    Online Schema Change Toolsadvanced
    Backfill Strategyadvanced
    Lock Analysis for DDLadvanced
    Migration Rollback Designadvanced
    Application and Schema Version Skewadvanced

    Hands-on projects

    1. 01Rename a column in a live system using four separate deployments, with the application working correctly after each one
    2. 02Backfill a hundred million rows in batches without pushing replication lag past your alert threshold
    3. 03Identify which DDL statements take an exclusive lock on your database version, and test the two you were unsure about
    4. 04Run an online schema change tool against a large table and measure its overhead against a direct ALTER
    5. 05Design a migration that can be rolled back after deployment, and state the point at which rollback stops being possible

  7. Capacity and Connection Management

    Databases fail on connections and IOPS long before they run out of CPU, and the failure mode is a thundering herd of application retries making it worse. Done when you can state your database's maximum sustainable connection count and write throughput from measurement rather than from the vendor's datasheet, and have an alert that fires before either is reached.

    3-4 weeks
    6 Skills
    Connection Pool Sizing
    IOPS and Throughput Planning
    Growth Forecasting
    Load Testing Databases
    Retry and Backoff Design
    Resource Saturation Alerting
    Show details, projects and resources

    Skills you'll master

    Connection Pool Sizingadvanced
    IOPS and Throughput Planningadvanced
    Growth Forecastingintermediate
    Load Testing Databasesadvanced
    Retry and Backoff Designadvanced
    Resource Saturation Alertingintermediate

    Hands-on projects

    1. 01Load test until the database degrades and record which resource saturated first — it is rarely the one you expected
    2. 02Size a connection pool from measurement, then prove the number by testing above and below it
    3. 03Forecast storage and IOPS growth twelve months out from real data, and state which assumption breaks the forecast
    4. 04Reproduce a retry storm and fix it with jittered backoff, measuring recovery time before and after
    5. 05Write saturation alerts that fire with enough lead time to act, and verify the lead time by triggering one

  8. Data Integrity and Corruption Detection

    Corruption is the failure mode that does not page anyone: it spreads into backups while every dashboard stays green, and gets discovered months later by a customer. Done when a checksum or consistency check runs on a schedule against real data, and you have seen it detect a fault you introduced deliberately rather than trusting that it would.

    3-4 weeks
    6 Skills
    Checksums and Page Verification
    Logical Consistency Checking
    Constraint and Foreign Key Design
    Replica Divergence Detection
    Silent Corruption Response
    Audit Trails
    Show details, projects and resources

    Skills you'll master

    Checksums and Page Verificationadvanced
    Logical Consistency Checkingadvanced
    Constraint and Foreign Key Designintermediate
    Replica Divergence Detectionadvanced
    Silent Corruption Responseadvanced
    Audit Trailsintermediate

    Hands-on projects

    1. 01Enable page checksums, corrupt a page deliberately, and confirm the database detects it rather than serving the bad data
    2. 02Compare a primary and replica row by row and find the divergence you introduced on purpose
    3. 03Write a scheduled logical consistency check for an invariant your schema cannot express as a constraint
    4. 04Trace how a corrupted page would propagate into your backups, and state how far back you would need to go
    5. 05Add constraints to a table that permits invalid states today, and record how many existing rows violate them

  9. Databases on Kubernetes

    Running stateful workloads on an orchestrator designed to reschedule things freely is the hardest operational problem in this roadmap, and the honest answer is sometimes not to. Done when you have run a database on Kubernetes through a node failure and a rolling upgrade without data loss, and can argue in writing when a managed service would have been the better call.

    4-5 weeks
    6 Skills
    StatefulSets and Persistent Volumes
    Database Operators
    Storage Classes and IOPS on Kubernetes
    Pod Disruption Budgets
    Backup in a Kubernetes Context
    Managed vs Self-hosted Decision
    Show details, projects and resources

    Skills you'll master

    StatefulSets and Persistent Volumesadvanced
    Database Operatorsadvanced
    Storage Classes and IOPS on Kubernetesadvanced
    Pod Disruption Budgetsintermediate
    Backup in a Kubernetes Contextadvanced
    Managed vs Self-hosted Decisionadvanced

    Hands-on projects

    1. 01Deploy a replicated database with an operator, then delete the primary pod and record what the operator did and how long it took
    2. 02Drain a node running a database pod and measure the disruption from the client's point of view
    3. 03Configure pod disruption budgets that survive a cluster upgrade, and verify by performing one
    4. 04Benchmark the same database on a persistent volume and on local storage, and record the durability trade-off
    5. 05Write the decision record for self-hosting versus a managed service, costed in engineering hours per month

  10. Data Reliability at Scale

    At scale the job stops being one database and becomes a fleet plus the humans who depend on it: RTO and RPO become contracts other teams design against, and self-service beats being a bottleneck. Done when a team other than yours provisions a compliant database without you in the loop, and an incident is handled using a runbook someone else wrote.

    4-5 weeks
    7 Skills
    RTO and RPO as Contracts
    Fleet Management and Automation
    Self-service Provisioning
    Database SLOs
    Incident Response for Data
    Runbook Authorship
    Sharding and Horizontal Scale
    Show details, projects and resources

    Skills you'll master

    RTO and RPO as Contractsadvanced
    Fleet Management and Automationadvanced
    Self-service Provisioningadvanced
    Database SLOsadvanced
    Incident Response for Dataadvanced
    Runbook Authorshipintermediate
    Sharding and Horizontal Scaleadvanced

    Hands-on projects

    1. 01Publish RTO and RPO per tier and get a dependent team to design against those numbers rather than assume zero
    2. 02Automate provisioning so another team can create a compliant database without you, then watch them use it unaided
    3. 03Define database SLOs that reflect what users experience rather than what is easy to measure, and report a month against them
    4. 04Write a runbook for the most likely data incident and have someone else execute it in a drill while you stay silent
    5. 05Design a sharding strategy for a table that has outgrown one machine, including how a resharding operation would run
    6. 06Run a game day on a data-loss scenario and record which step of the response was slowest

What the job is actually like

Day to day
Quieter than the incident stories suggest, because most of the week goes on preventing the incident. That means reviewing schema migrations before they lock a table in production, watching replication lag drift and working out why, and running the restore that proves the backup is a backup. There is steady query work, usually arriving as a complaint that something is slow. What surprises people is how much of it is conversation: talking a team out of a design that will be unfixable at ten times the row count is worth more than any tuning you do afterwards. When it does go wrong it goes wrong with the data, which does not roll back casually.
The interview
Expect a scenario rather than trivia — the primary has failed over, replication is behind, what do you do and in what order. Depth in one engine beats shallowness across five, but you will be asked to reason about a database you have never used, because the concepts are what is being tested. A migration exercise is common: change this schema on a large live table without downtime. Backup and restore comes up in nearly every process, and the expected answer describes a restore you have personally performed and timed. Query rounds ask you to read a plan, not to recite index types.
How people get in
Mostly from operations rather than from development. Site reliability and infrastructure engineers arrive with production instinct and have to learn where databases refuse to behave like stateless services — the reliability path shares this roadmap's data phase and diverges after it. Traditional database administrators arrive with deep engine knowledge and have to pick up automation, code review and cloud primitives. Backend engineers who became the person who understood the slow query are the third group and often the strongest, because they know why the schema looks like that. What does not transfer is fixing production by hand.
After senior
The specialist fork keeps going deeper and stays well paid, because the supply of people who can be trusted with stateful systems does not grow. A broader fork moves into data platform or infrastructure work, where databases become one component of something larger. Some move into the reliability path proper, trading depth for breadth. There is also a consulting route that is unusually viable here: organisations will pay well for a few weeks of someone who has restored a corrupted cluster before. Titles vary more than in most roles, so read the responsibilities rather than the name on the posting.
Why people leave
The first is being the only person who understands the database. It feels like job security and is actually a trap — unpromotable, unable to take a holiday, and the organisation never builds a practice around you. Insist on writing it down. The second is a job that is restores and ticket queues with no authority over design, where you inherit every decision and own every consequence; ask in the interview whether database review happens before a migration ships. The third is technology narrowing: a decade on one engine is worth far less than a decade on the problems, and the problems are portable.

Frequently asked questions

Related certifications

Related roadmaps