How to Build a Job Queue in .NET with PostgreSQL SKIP LOCKED: A Real-World Walkthrough
Every .NET team eventually needs to run work outside the request cycle: send the welcome email, generate the invoice PDF, push the webhook, re-index the search document. The default instinct is to reach for a message broker or Hangfire. But if your app already talks to PostgreSQL, you are one SQL clause away from a durable, concurrent work queue with no extra infrastructure. That clause is FOR UPDATE SKIP LOCKED, and a PostgreSQL job queue built on it in .NET is one of the most underrated patterns I keep coming back to in production. It survives restarts, scales across worker instances, and never hands the same job to two workers.
I have shipped this pattern on systems processing millions of jobs a day, and the thing that surprises people most is how little code it takes. If you want the complete, annotated implementation alongside this article - the full worker, the retry and dead-letter logic, a reaper for stuck jobs, and an integration test suite that hammers it with concurrent consumers - I keep the ready-to-run version on Patreon, so you can clone it and adapt it to your own schema in an afternoon.
There is one part that trips up almost everyone the first time: wiring the claim loop into a resilient background worker with a correctly scoped DbContext. Getting a scoped DbContext into a singleton BackgroundService, running it with graceful shutdown, and layering retries on top is exactly what Chapter 12 (Background Jobs and Long-Running Tasks) of the Zero to Production course walks through inside a complete ASP.NET Core API, so the hosting side is never the thing that bites you.
This walkthrough covers the business problem, the design decisions, a working implementation with EF Core 10, the trade-offs I have learned the hard way, and what to reach for when this pattern stops being enough.
The Problem: One Job, Many Workers, No Duplicates
Picture a jobs table and three worker processes polling it. The naive query is SELECT * FROM jobs WHERE status = 'pending' LIMIT 1. Run three copies of that at once and all three read the same row, all three process it, and your customer gets three welcome emails. Add a SELECT ... FOR UPDATE to lock the row and you fix the duplication - but now workers two and three block, waiting for worker one to release the lock. Your concurrency just collapsed to one job at a time.
The requirement is subtle but strict:
Exactly one worker may claim a given job.
A worker must never wait on a job another worker already holds.
The claim must survive a crash - if a worker dies mid-job, the job must become available again.
FOR UPDATE SKIP LOCKED solves the first two in a single clause. The third is a design decision we make on top of it.
What Is FOR UPDATE SKIP LOCKED in PostgreSQL?
FOR UPDATE SKIP LOCKED is a row-locking clause that tells PostgreSQL to lock the rows a query selects and silently skip any row already locked by another transaction, instead of waiting for it.
In practice that means:
Worker A runs the query, locks row 1, and starts processing.
Worker B runs the same query at the same moment. Row 1 is locked, so PostgreSQL skips it and hands Worker B row 2.
No worker ever blocks, and no two workers ever get the same row.
The lock is held until the transaction commits or rolls back. SKIP LOCKED has been available since PostgreSQL 9.5, so any supported version has it. This is the same mechanism the PostgreSQL documentation describes for the locking clause of SELECT, and it is battle-tested well beyond .NET.
Designing the Jobs Table
A queue table needs enough columns to track state, ordering, retries, and delayed execution. This is the schema I start with:
CREATE TABLE jobs (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
type text NOT NULL,
payload jsonb NOT NULL,
status text NOT NULL DEFAULT 'pending',
attempts int NOT NULL DEFAULT 0,
run_at timestamptz NOT NULL DEFAULT now(),
locked_at timestamptz
);
-- Index only the rows workers actually query
CREATE INDEX ix_jobs_ready ON jobs (run_at) WHERE status = 'pending';
Two decisions matter here. First, run_at gives you delayed jobs and retry backoff for free: a worker only picks up rows where run_at <= now(). Second, the partial index WHERE status = 'pending' keeps the index tiny and fast even when the table holds millions of completed rows, because it only indexes the rows a worker can actually claim. That single line is the difference between a queue that stays fast at scale and one that degrades as history piles up.
Claiming a Job with EF Core 10
EF Core has no LINQ operator for a locking hint, so the claim query has to be raw SQL. The good news: results from FromSql are fully change-tracked, so once you claim a row you update it like any other entity. This snippet requires EF Core 10 and the Npgsql provider (Microsoft.EntityFrameworkCore and Npgsql.EntityFrameworkCore.PostgreSQL, both 10.x):
await using var tx = await db.Database.BeginTransactionAsync(ct);
// Keep LIMIT inside the raw SQL. Do NOT chain FirstOrDefaultAsync:
// EF Core wraps composed LINQ in a subquery, and PostgreSQL
// rejects FOR UPDATE inside a subquery.
var claimed = await db.Jobs
.FromSql($"""
SELECT * FROM jobs
WHERE status = 'pending' AND run_at <= now()
ORDER BY run_at
LIMIT 1
FOR UPDATE SKIP LOCKED
""")
.ToListAsync(ct);
var job = claimed.FirstOrDefault();
if (job is null)
{
await tx.CommitAsync(ct);
return; // nothing ready right now
}
job.Status = "processing";
job.LockedAt = DateTimeOffset.UtcNow;
await db.SaveChangesAsync(ct);
await tx.CommitAsync(ct); // release the row lock immediately
The comment above is not decorative - it is the single most common mistake I see. Developers write .FromSql(...).FirstOrDefaultAsync() because that is the idiomatic EF Core way, EF Core wraps the raw SQL in a SELECT ... FROM (your-sql) LIMIT 1 subquery, and PostgreSQL throws FOR UPDATE is not allowed with set operations or a locking-clause error. Put the LIMIT in your SQL, materialize with ToListAsync, and take the first result in C#. Note also that FromSql uses an interpolated string here purely for the EF Core API shape - there is no user input, so there is no injection surface, but EF Core would parameterize any interpolated value anyway. See the Microsoft guidance on raw SQL queries in EF Core for the composition rules.
Claim-Then-Process vs Lock-and-Hold
Notice that the snippet above commits the transaction before doing any real work. This is deliberate, and it is the design decision that separates a toy queue from a production one.
Lock-and-hold: begin transaction, claim the row, do the work, mark it done, then commit. The row lock is held for the entire duration of the job. Simple, but a 30-second job holds a row lock and a database connection for 30 seconds. That does not scale.
Claim-then-process (recommended): begin transaction, claim the row by setting
status = 'processing', commit immediately to release the lock, then do the work outside any transaction, and finally mark the jobdoneorfailed. Locks are held for milliseconds regardless of how long the job runs.
Claim-then-process introduces one new failure mode: if a worker crashes after claiming but before finishing, the job is stranded in processing forever. That is what the reaper solves.
Handling Crashes, Retries, and Poison Jobs
A production queue needs three safety nets, and none of them require anything exotic.
A reaper for stuck jobs. A small periodic query resets rows that have been processing for longer than any real job should take. Because you stored locked_at at claim time, this is one statement:
UPDATE jobs
SET status = 'pending', locked_at = NULL
WHERE status = 'processing' AND locked_at < now() - interval '5 minutes';
Retry with backoff. When a handler throws, do not lose the job. Increment attempts, push run_at into the future, and set it back to pending:
job.Attempts++;
job.Status = "pending";
job.RunAt = DateTimeOffset.UtcNow + TimeSpan.FromSeconds(Math.Pow(2, job.Attempts));
await db.SaveChangesAsync(ct);
A dead-letter state for poison jobs. A job that fails every time will otherwise retry forever. Once attempts crosses a ceiling, move it to a terminal dead status so a human can inspect it rather than letting it churn.
There is one truth you cannot design away: with claim-then-process, a job can run twice - a worker can finish the work and then crash before marking it done, so the reaper hands it back out. Your handlers must be idempotent. Sending an email? Record a sent-message key first. Charging a card? Use an idempotency key. This is the same discipline the Outbox Pattern relies on, and it is non-negotiable for any at-least-once system.
Running the Worker as a BackgroundService
The claim loop needs a host. A BackgroundService with a PeriodicTimer is the clean way to poll, and because BackgroundService is a singleton you must resolve a fresh scoped DbContext per tick through IServiceScopeFactory:
public sealed class JobWorker(IServiceScopeFactory scopes) : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken ct)
{
using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1));
while (await timer.WaitForNextTickAsync(ct))
{
await using var scope = scopes.CreateAsyncScope();
var db = scope.ServiceProvider.GetRequiredService<JobsDbContext>();
await ClaimAndRunOneAsync(db, ct);
}
}
}
Register it with builder.Services.AddHostedService<JobWorker>() and run as many instances of your app as you like - SKIP LOCKED guarantees they never collide. If one-second polling feels wasteful, PostgreSQL's LISTEN/NOTIFY lets a producer wake workers the instant a job is enqueued, turning the poll into a fallback rather than the primary trigger. If you want a deeper tour of the hosting choices - IHostedService vs BackgroundService vs Worker Service, and when each fits - our guide on background services in .NET 10 covers the trade-offs.
When Should You Use a Database Queue Instead of a Message Broker?
Reach for a PostgreSQL job queue when the queue is an implementation detail of one application and you value operational simplicity over raw throughput.
A database queue is the right call when:
You already run PostgreSQL and do not want to operate RabbitMQ, Kafka, or a cloud broker just for background work.
You want jobs to commit transactionally with your business data - enqueue the job in the same
SaveChangesAsyncthat writes the order, so a job never references data that was rolled back.Your volume is in the thousands-to-low-millions of jobs per day, comfortably within a single well-indexed table.
Reach for a dedicated broker or a tool like Hangfire when you need cross-service fan-out, guaranteed ordering across partitions, sustained throughput beyond what one database can absorb, or features like a management dashboard out of the box. If you are weighing those options, our comparison of background jobs at scale with Hangfire, Quartz, and Azure Functions maps the landscape. The honest trade-off I have lived with: the database queue costs you nothing to operate and everything is transactional, but it puts queue load on the same database serving your API, so watch connection counts and keep that partial index lean.
What to Do Next
Start small. Add the jobs table, wire one BackgroundService, and move a single existing background task - the welcome email is a classic first candidate - onto it. Confirm the claim-then-process flow, add the reaper, then make your handlers idempotent before you trust it with anything that touches money or external systems. Once the shape is solid, layer in retries with backoff and a dead-letter status. That progression, from a single claim to a resilient multi-worker queue, is the whole pattern.
FAQ
What Is FOR UPDATE SKIP LOCKED Used For in a Job Queue?
It lets multiple workers pull jobs from the same table concurrently without ever grabbing the same row or blocking each other. Each worker locks the row it claims and PostgreSQL skips rows already locked by other workers, so throughput scales with the number of workers instead of serializing to one.
Is a PostgreSQL Job Queue with SKIP LOCKED Safe for Multiple Workers?
Yes. SKIP LOCKED guarantees that a locked row is invisible to every other worker's claim query, so exactly one worker can claim a given job. The only caveat is crash recovery: because you release the lock before processing (claim-then-process), a job can run more than once if a worker dies mid-job, which is why handlers must be idempotent.
Can I Use FOR UPDATE SKIP LOCKED with EF Core, or Do I Need Raw SQL?
You need raw SQL for the claim query - EF Core has no LINQ operator for a locking hint. Use FromSql with the FOR UPDATE SKIP LOCKED clause and keep LIMIT inside the SQL rather than chaining FirstOrDefaultAsync, because EF Core wraps composed queries in a subquery and PostgreSQL rejects FOR UPDATE inside a subquery. The rows you get back are still change-tracked, so updating status works normally.
When Should I Use a Database Queue Instead of Hangfire or a Message Broker?
Use a database queue when you already run PostgreSQL, want jobs to commit transactionally with your business data, and your volume fits a single indexed table. Choose Hangfire or a broker when you need a dashboard, cross-service messaging, guaranteed ordering, or throughput beyond what one database comfortably handles.
How Do I Handle Stuck or Failed Jobs in a SKIP LOCKED Queue?
Store a locked_at timestamp when you claim a job and run a periodic reaper that resets any row stuck in processing past a timeout back to pending. For failures, increment an attempts counter and push run_at into the future for exponential backoff, and once attempts cross a ceiling, move the job to a terminal dead status for manual inspection.
About the Author
I'm Celin Daniel, Co-founder of Coding Droplets. I've been building .NET and ASP.NET Core systems in production for 13+ years - APIs, distributed backends, enterprise platforms. Everything I write here comes from real shipping experience: patterns that held up, trade-offs that bit us, and lessons learned the hard way.
GitHub: codingdroplets
YouTube: Coding Droplets
Website: codingdroplets.com
