The Pipes and Filters Pattern in ASP.NET Core: When to Use It and How
The Pipes and Filters pattern is one of those architectural ideas that shows up everywhere in .NET — in ASP.NET Core's middleware stack, in MediatR's pipeline behaviors, in System.Threading.Channels processing loops — yet most teams never deliberately name it when they use it, or reach for it deliberately when they need it. This guide closes that gap.
If you want to see how these patterns work inside a complete production-grade ASP.NET Core codebase — with pipeline behaviors, background processing stages, and all of it wired together — the full implementation is available on Patreon, with annotated source code that maps directly to what enterprise teams actually ship.
Understanding processing pipelines in isolation is useful — seeing them as part of a complete production API alongside CQRS, Clean Architecture, and background task orchestration is what makes the design decisions click. That is exactly what Chapter 12 of the ASP.NET Core Web API: Zero to Production course covers — with source code you can run immediately.
What Problem Does the Pipes and Filters Pattern Solve?
The Pipes and Filters pattern structures a processing task as a sequence of independent, composable stages — each stage (a filter) receives input, transforms or evaluates it, and passes the result to the next stage via a pipe. No stage needs to know about the stages before or after it. Each stage has a single, well-defined responsibility.
The pattern addresses a recurring problem: when a task involves multiple distinct processing steps that can be independently tested, reordered, enabled, or replaced without the other steps knowing. Tangled, monolithic service methods that do validation, enrichment, transformation, and persistence in one continuous flow are the symptom this pattern is designed to fix.
In ASP.NET Core, the middleware pipeline is a first-class, built-in implementation of Pipes and Filters. Every Use, UseWhen, or Map call registers a filter. The request flows through them in registration order. Each middleware calls next() to pass control downstream and can act again on the way back. That is the pattern — applied at the HTTP layer.
The Two Faces of Pipes and Filters in .NET
In practice, the pattern shows up in two distinct forms inside .NET applications.
The Request Pipeline (Framework-Level)
ASP.NET Core's IMiddleware and RequestDelegate chain are the canonical implementation. The framework owns the pipe; you implement the filters. You add cross-cutting concerns — authentication, rate limiting, correlation ID propagation, exception handling — as discrete middleware components.
The filter in this context is IMiddleware or the convention-based InvokeAsync(HttpContext context, RequestDelegate next) signature. Each component has access to HttpContext, can short-circuit the pipeline (by not calling next), and can observe or modify both the request and the response.
Custom Domain Pipelines (Application-Level)
The second form appears when you build your own pipeline for domain-level processing: a document ingestion flow that validates → enriches → classifies → persists; a payment workflow that checks limits → applies fraud rules → calls the processor → emits an event; a data transformation chain that normalises → validates → maps → stores.
Here you implement the pipe yourself, and MediatR's IPipelineBehavior<TRequest, TResponse> is the most natural way to do it in a Clean Architecture + CQRS setup. Each behavior is a filter; MediatR is the pipe. System.Threading.Channels serves the same role in producer-consumer processing loops where you need backpressure, concurrency control, and stage isolation.
When to Use the Pipes and Filters Pattern
Apply the pattern when the following conditions hold:
The task has multiple distinct stages. If a single service method performs validation, enrichment, transformation, and persistence as one sequential block, each step is an implicit filter. Making them explicit — and decoupled — yields components you can test independently and reorder without side effects.
Stages have different operational characteristics. Some stages are CPU-bound (parsing, hashing), others are I/O-bound (database writes, external API calls), others are stateless (validation). When stages differ in latency profile, error handling strategy, or retry policy, separating them into discrete components lets you apply the right operational model to each.
The pipeline needs to evolve without coordinated changes. New business requirements often mean new stages — a compliance check, a new enrichment source, an audit event. A Pipes and Filters structure absorbs new stages at the insertion point without modifying the existing stages around them.
Cross-cutting concerns must not bleed into business logic. Logging, correlation tracking, caching, validation, and performance measurement belong in the pipeline, not in the handlers. Pipeline behaviors (MediatR) and middleware (ASP.NET Core) enforce this boundary structurally — it is architecturally impossible for a handler to skip the logging behavior if the behavior wraps every command.
Stages can be reused across multiple pipelines. A fraud-check stage that reads from a shared rules engine, or a normalisation stage that applies canonical formatting, should be written once and composed into any pipeline that needs it — not copy-pasted.
When Not to Use It
The pattern introduces indirection. When that indirection does not pay off, avoid it.
When the task is genuinely simple. A CRUD endpoint that reads from the database and returns a DTO does not benefit from a four-stage pipeline. The overhead of designing, naming, and wiring discrete stages is not justified for straightforward transformations.
When stages are tightly coupled by shared state. Pipes and Filters works because each stage is independent. If your stages share a mutable context object that grows as it passes through, you have a different pattern — the Chain of Responsibility or a workflow engine — and you should name it as such.
When ordering is non-trivial and frequently changes. If stage ordering has complex conditional logic ("run stage B only after stage A, but only if stage C did not short-circuit"), you are describing an orchestrated workflow, not a linear pipeline. A workflow engine or saga handles that more explicitly.
When you need guaranteed stage isolation in distributed systems. A single-process MediatR pipeline shares the same process and transaction scope. If stages genuinely need to run in separate services, survive process restarts independently, or scale independently, the Competing Consumers or Saga patterns are more appropriate.
How the Pattern Is Expressed in ASP.NET Core
Middleware as Pipes and Filters
The simplest expression of the pattern in ASP.NET Core is the middleware pipeline registered in Program.cs. The registration order is the execution order. Short-circuiting is explicit: a middleware that does not call next terminates the pipeline at that stage.
The key design principle here is the single-responsibility constraint on each component. A rate limiting middleware should do exactly that — apply a rate limit — and nothing else. If it starts checking authentication state to decide whether to rate limit differently, it has absorbed a concern that belongs in a dedicated component earlier or later in the pipeline.
MediatR Pipeline Behaviors
For application-layer pipelines, IPipelineBehavior<TRequest, TResponse> is the idiomatic filter interface. Validation behavior, logging behavior, caching behavior, and transaction behavior all implement this interface. MediatR resolves them in dependency injection registration order and executes them before invoking the handler.
The important constraint: pipeline behaviors should be generic where possible. A ValidationBehavior<TRequest, TResponse> that invokes all registered IValidator<TRequest> instances from DI is a reusable filter for every command and query in the application. A behavior written to handle only one specific request type is not a pipeline behavior — it is business logic that belongs in the handler.
System.Threading.Channels for Async Stage Pipelines
When stages need to process work asynchronously with backpressure, concurrency limits, and stage-local consumers, System.Threading.Channels models the pipe explicitly. Each stage reads from one channel and writes to the next. BoundedChannel provides the backpressure mechanism — if a downstream stage is slow, the bounded buffer prevents unbounded memory growth upstream.
This form is appropriate for document ingestion, event stream processing, or any fan-in/fan-out processing requirement where MediatR's synchronous (per-request) model would create blocking or excessive thread contention.
Trade-offs and Anti-Patterns
The Trade-offs
| Concern | Benefit | Cost |
|---|---|---|
| Testability | Each stage tests in isolation | More test surface area to maintain |
| Extensibility | New stages insert without modifying existing ones | Registration and ordering must be actively managed |
| Observability | Each stage can be independently logged and traced | Per-stage telemetry requires explicit instrumentation |
| Decoupling | Stages do not depend on each other | Shared context objects can re-introduce coupling |
Anti-Patterns to Avoid
The God Filter. A pipeline behavior or middleware that handles validation, enrichment, caching, and audit logging in a single class. It is indistinguishable from a monolithic service method, just with different plumbing.
Hidden side effects between stages. Stages that write to ambient state — a shared dictionary on the context, a ThreadLocal, an unscoped service — create invisible ordering dependencies. If stage B fails because stage A did not populate a key on a shared object, the pipeline is coupled in ways the type system cannot express.
Overusing the middleware pipeline for business logic. UseWhen and conditional middleware are powerful, but complex branching in Program.cs is a sign that the logic belongs in application-layer handlers, not in the HTTP pipeline. The middleware pipeline should deal with infrastructure concerns. Business rules belong in MediatR handlers and domain models.
Ignoring the ordering contract. AddAuthentication and UseAuthentication must precede UseAuthorization. UseRouting must precede UseEndpoints. The middleware pipeline has implicit ordering contracts that are not enforced by the compiler. Violating them produces bugs that only appear in production under specific conditions. Document the required ordering explicitly in a comment on the registration block.
Decision Matrix
| Scenario | Recommended Approach |
|---|---|
| HTTP cross-cutting concerns (auth, rate limiting, CORS, logging) | ASP.NET Core middleware pipeline |
| Application-layer cross-cutting concerns (validation, caching, transactions) | MediatR IPipelineBehavior<TRequest, TResponse> |
| Async multi-stage background processing with backpressure | System.Threading.Channels stage pipeline |
| Distributed multi-service workflows with compensation | Saga pattern |
| Simple CRUD endpoint | No pipeline — direct handler |
What to Do with Existing Code
If you have a large service class with tangled validation, enrichment, and persistence logic, refactoring to an explicit pipeline is a good investment — but do it in stages.
Start by extracting cross-cutting concerns into MediatR behaviors. Validation is the easiest first step. A ValidationBehavior applied globally immediately removes validation code from every handler and makes the constraint explicit and reusable.
Next, identify the stages hidden inside your handlers. If a handler fetches an entity, applies a business rule, raises a domain event, and saves — those are four stages, three of which (fetch, raise event, save) are infrastructure-layer concerns that belong in behaviors or event handlers, not mixed with the business rule itself.
Use System.Threading.Channels only when you have a genuine async processing requirement with observable backpressure needs. Do not introduce channel-based pipelines as a default architecture — they carry significant operational complexity and are worth it only when the concurrency and backpressure semantics are genuinely needed.
Internal to Coding Droplets, we have written about the related 7 Common ASP.NET Core Middleware Mistakes and How to Fix Them and the CQRS and MediatR in ASP.NET Core: Enterprise Decision Guide — both worth reading alongside this guide for a complete picture of pipeline design in .NET.
For the authoritative reference on the pattern's theory, the Pipes and Filters pattern on the Azure Architecture Center is the canonical starting point, and Microsoft's ASP.NET Core Middleware documentation covers the framework implementation in depth.
☕ Prefer a one-time tip? Buy us a coffee — every bit helps keep the content coming!
FAQ
What is the Pipes and Filters pattern in ASP.NET Core? The Pipes and Filters pattern structures a processing task as a sequence of independent stages connected by pipes. In ASP.NET Core, the middleware pipeline is the built-in implementation at the HTTP layer. At the application layer, MediatR pipeline behaviors implement the same pattern for command and query processing.
How is the Pipes and Filters pattern different from the Chain of Responsibility? Chain of Responsibility passes a request through handlers where each handler decides whether to process it or forward it — typically only one handler acts. Pipes and Filters passes data through all stages sequentially, with each stage performing its own transformation or evaluation. In ASP.NET Core middleware, both handlers and short-circuiting middleware both exist, but the overall structure is Pipes and Filters.
When should I use MediatR pipeline behaviors vs ASP.NET Core middleware? Use ASP.NET Core middleware for HTTP-level infrastructure concerns — authentication, rate limiting, CORS, exception handling, compression, and request/response logging at the transport level. Use MediatR pipeline behaviors for application-layer cross-cutting concerns — validation, caching, transaction scope, and audit logging at the business operation level. The key signal: if the concern requires knowledge of HttpContext, it belongs in middleware; if it requires knowledge of the command or query type, it belongs in a pipeline behavior.
Can I build custom Pipes and Filters pipelines without MediatR? Yes. The pattern does not require a framework. A simple interface such as IPipelineFilter<T> with an Execute(T context, Func<T, Task> next) signature, combined with a builder that chains filters in registration order, is sufficient. MediatR provides this out of the box for CQRS workloads, and System.Threading.Channels provides it for async producer-consumer flows. For lightweight scenarios, a hand-rolled pipeline with no third-party dependency is a reasonable choice.
How does the Pipes and Filters pattern affect testability? It significantly improves testability. Each filter or behavior can be tested in isolation with a unit test. A validation behavior needs only a set of validators and a fake next delegate. A middleware component needs only an HttpContext and a RequestDelegate. There is no need to construct the entire pipeline to test a single stage — which contrasts sharply with monolithic service methods where a test must trigger all logic simultaneously.
What is the right number of stages in a pipeline? There is no fixed answer, but a useful heuristic: each stage should have a name that clearly describes its single responsibility. If you cannot name a stage without using "and", split it. In practice, MediatR pipelines with three to six behaviors (logging, validation, caching, transaction, error handling, audit) are common in production systems. Middleware pipelines in ASP.NET Core often have ten or more components — each provided either by the framework or as a named, composable component.
Should I use System.Threading.Channels or a message broker for stage pipelines? Use System.Threading.Channels when all stages run in the same process and you need in-process backpressure, concurrency control, and stage isolation without external dependencies. Use a message broker (RabbitMQ, Azure Service Bus, Kafka) when stages need to run in separate processes or services, survive process restarts independently, or scale horizontally at the stage level. Channels are an in-process primitive; brokers are distributed infrastructure. Do not reach for a broker to solve a problem that BoundedChannel<T> handles in ten lines.
