How to Build Incremental RAG Indexing in .NET: A Real-World Walkthrough
Incremental RAG indexing is what separates a demo knowledge base from one you can still trust six months later. The first ingestion run is easy: read every document, chunk it, embed it, write it to a vector store. The trouble starts on day two, when a refund policy changes, a product is discontinued, and an FAQ page gets deleted, and your .NET API keeps answering from the old versions because nothing ever told the index.
Every snippet in this walkthrough compiles against Microsoft.Extensions.VectorData.Abstractions 10.10.0, and I ran the full sync flow against the InMemory connector - an unchanged document, an edit that shrinks the chunk count, and a deletion at the source - before writing it up. If you want the complete implementation with the registry, the reconciliation job, and the tests wired into a running API, that is the kind of production-ready code that lives on Patreon.
Keeping the index in sync is the last step of a proper ingestion pipeline, not an afterthought bolted on later. Chapter 9 of AI-Powered .NET APIs builds that pipeline in C# - chunking, embedding, storing chunks with source metadata, an admin ingest endpoint, and idempotent re-ingestion driven by a BackgroundService - inside one running ASP.NET Core support API.
The Business Problem: A Knowledge Base That Changes Every Day
Picture a support API grounded in 5,000 help-center articles and policy documents. The content team edits around 50 of them a day and retires a few each week.
The obvious answer is a nightly full re-index. At an average of 2,000 tokens per document, that is 10 million tokens of embedding work every night to pick up roughly 1% of changes, and answers can still be up to a day out of date. The second obvious answer, "just upsert the document that changed", creates five failure modes of its own:
Stale answers. The document changed, but its old chunks are still in the index and still rank.
Orphaned chunks. A document shrinks from 12 chunks to 8, and with position-based keys, chunks 8 through 11 survive forever.
Ghost answers. A deleted document stays retrievable and gets cited. This is the dangerous one, because content is usually deleted for a reason: a withdrawn offer, a legal request, an error.
The empty window. Delete-then-reinsert leaves a gap in which searches for that document return nothing.
Mixed embedding spaces. After a model change, vectors from two models share one collection and similarity scores stop meaning anything.
What Is Incremental RAG Indexing?
Incremental RAG indexing updates a vector index by processing only the documents that changed since the last run. It detects changes with a content fingerprint, replaces a changed document's chunks as a unit, and removes the chunks of documents that no longer exist at the source. Unchanged documents cost nothing: no chunking, no embedding calls, no writes.
| Concern | Mechanism | Failure mode it prevents |
|---|---|---|
| Change detection | Content hash stored in a source registry | Re-embedding unchanged documents |
| Replacement | Write the new chunks, then delete the old ones | Stale answers, orphaned chunks, the empty window |
| Deletion | Reconcile the registry against the source | Ghost answers |
| Pipeline changes | Pipeline version inside the fingerprint | Mixed embedding spaces |
Design Decisions
Keep a Source Registry Outside the Vector Store
Store one row per source document: its ID, content hash, chunk count, indexed time, and a deleted marker. It belongs in your relational database, not the vector store, because the question reconciliation asks - "what have I indexed that no longer exists?" - is a set difference, and vector store APIs are shaped for similarity search with top-N limits. The registry doubles as the audit trail when someone asks why the assistant quoted a policy.
Fingerprint the Content Plus the Pipeline Version
Hash the normalized text together with a pipeline version string that names the chunker, chunk size, overlap, and embedding model. Change any of those and every fingerprint changes on purpose, which turns "we switched models" into an ordinary, controlled rebuild instead of a silent mix of vector spaces.
Normalize before hashing. Exports that stamp a "generated at" time or reflow whitespace make every document look changed on every run, and you are back to paying for a full re-index without noticing.
Replace by Writing New Chunks Before Deleting Old Ones
Give each chunk a key built from the source ID, a prefix of the content hash, and its position. A new version then never collides with the old one, so you can write the new chunks first and delete the old ones second. There is no empty window. The trade-off is a few seconds during which both versions are searchable, which is acceptable for most knowledge bases. If it is not for yours, deduplicate results by source ID at query time and keep only the hash the registry considers current.
Implementation Walkthrough
The Chunk Record
public sealed class KnowledgeChunk
{
[VectorStoreKey] public string Key { get; set; } = ""; // "{sourceId}:{hash12}:{index}"
[VectorStoreData(IsIndexed = true)] public string SourceId { get; set; } = "";
[VectorStoreData(IsIndexed = true)] public string ContentHash { get; set; } = "";
[VectorStoreData] public string Text { get; set; } = "";
[VectorStoreVector(768, DistanceFunction = DistanceFunction.CosineDistance)] public ReadOnlyMemory<float> Embedding { get; set; }
}
SourceId and ContentHash are indexed data properties because the sync process filters on both. The 768 dimensions match nomic-embed-text; use whatever your embedding model produces.
Change Detection and Replacement
public async Task SyncAsync(SourceDocument doc, CancellationToken ct)
{
string hash = Fingerprint(Normalize(doc.Text)); // SHA-256 of pipeline version + normalized text
SourceEntry? entry = await registry.FindAsync(doc.Id, ct);
if (entry?.ContentHash == hash) return; // unchanged: zero embedding calls
IReadOnlyList<string> chunks = chunker.Chunk(doc.Text);
GeneratedEmbeddings<Embedding<float>> vectors = await embedder.GenerateAsync(chunks, cancellationToken: ct);
var records = chunks.Select((text, i) => new KnowledgeChunk
{
Key = $"{doc.Id}:{hash[..12]}:{i}", SourceId = doc.Id, ContentHash = hash,
Text = text, Embedding = vectors[i].Vector
});
await collection.UpsertAsync(records, ct); // 1. the new version becomes searchable
await DeleteChunksAsync(doc.Id, keepHash: hash, ct); // 2. then the old version disappears
await registry.SaveAsync(new SourceEntry(doc.Id, hash, chunks.Count, DateTimeOffset.UtcNow), ct);
}
The order of the last three lines is the whole design. The registry is updated last, so a crash between steps leaves the old fingerprint in place and the next run simply redoes the work. Every step is safe to repeat.
Deleting Stale Chunks by Filter
public async Task DeleteChunksAsync(string sourceId, string? keepHash, CancellationToken ct)
{
List<string> stale = await collection
.GetAsync(c => c.SourceId == sourceId && c.ContentHash != keepHash, top: 1_000, cancellationToken: ct)
.Select(c => c.Key)
.ToListAsync(ct);
if (stale.Count > 0)
await collection.DeleteAsync(stale, ct);
}
Passing keepHash: null deletes every chunk for a source, which is exactly what reconciliation needs. Two cautions: loop until the query returns nothing if a single document can exceed 1,000 chunks, and confirm your connector's filter translation supports inequality before relying on it. On .NET 10, Select and ToListAsync over the IAsyncEnumerable come from the framework's built-in async LINQ, with no extra package.
Reconciling Deletions at the Source
public async Task ReconcileDeletionsAsync(IReadOnlySet<string> liveIds, CancellationToken ct)
{
foreach (string goneId in await registry.ListActiveIdsNotInAsync(liveIds, ct))
{
await DeleteChunksAsync(goneId, keepHash: null, ct);
await registry.MarkDeletedAsync(goneId, ct);
}
}
liveIds comes from the source system: blob names, CMS entry IDs, file paths. This is the only mechanism that catches deletions, because a document that no longer exists never arrives in a change event you can process.
Running It: Events for Freshness, a Sweep for Correctness
Use two triggers. Change events from your CMS webhook or blob storage notifications flow into a Channel, and a hosted service drains it for near real-time updates:
public sealed class IndexSyncWorker(Channel<SourceDocument> changes, IServiceScopeFactory scopes) : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await foreach (SourceDocument doc in changes.Reader.ReadAllAsync(stoppingToken))
{
using IServiceScope scope = scopes.CreateScope();
await scope.ServiceProvider.GetRequiredService<IncrementalIndexer>().SyncAsync(doc, stoppingToken);
}
}
}
Then run a scheduled reconciliation sweep, hourly or nightly, that lists the source, syncs anything whose fingerprint differs, and removes anything that disappeared. Events give you freshness; the sweep gives you correctness when a webhook is dropped or a deployment eats a queue message. Route sensitive deletions such as legal takedowns through the event path immediately rather than waiting for the sweep, and make sure only one sync runs per source at a time, with a distributed lock if you run several instances.
What Happened When I Ran It
Here is the index state after each step of my test run against the InMemory connector, using a paragraph chunker and an embedder that counts its calls:
| Step | Chunks in index | Embedding calls | Texts embedded |
|---|---|---|---|
Initial sync: returns (3 paragraphs), shipping (2) |
5 | 2 | 5 |
Re-sync returns after a whitespace-only change |
5 | 2 | 5 |
Edit returns down to 2 paragraphs |
4 | 3 | 7 |
Reconcile after shipping is deleted at the source |
2 | 3 | 7 |
The second row is the point of the design: a re-sync of unchanged content costs nothing. The third row shows no orphans, because all three old returns chunks were replaced by two new ones with a new hash in their keys. The fourth shows the ghost-answer failure mode closed.
Where Microsoft.Extensions.DataIngestion Fits
Microsoft.Extensions.DataIngestion gives you readers, chunkers, enrichers, and a VectorStoreWriter<T> chained into an ingestion pipeline. It is still in preview (10.10.0-preview.1 at the time of writing), and the data ingestion overview on Microsoft Learn is the best starting point.
The writer already handles replacement well. VectorStoreWriterOptions.IncrementalIngestion defaults to true, and in the current source the writer looks up the chunks previously stored for a document through a documentid field, inserts the new chunks, and only then deletes the old ones. That is the same write-new-then-delete-old ordering described above.
What it does not decide for you is which documents to process. It re-chunks and re-embeds whatever you feed it, so unchanged documents still cost embedding calls, and it never sees documents that were deleted at the source. The registry, the fingerprint, and the reconciliation sweep still sit in front of it.
Trade-offs and Gotchas
An embedding model change needs a new collection. New dimensions cannot be written into the existing collection, and even identical dimensions from a different model are not comparable. Build a new collection in the background, backfill it, switch reads over, then drop the old one.
Hash content and metadata separately. If chunks carry permissions, tenant IDs, or categories for filtering, a permission change must update the chunks but should not trigger re-embedding. Fetch existing records with
IncludeVectors = trueand upsert the new metadata alongside the old vectors.Some stores are eventually consistent. A deleted key can reappear in a query for a short time, so bound any delete loop instead of spinning until empty.
Cap chunks per document. A malformed conversion that produces one chunk per line can turn a single document into thousands of embedding calls. Alert when a document's chunk count jumps.
Deletion latency is a compliance question. Decide how quickly removed content must stop being retrievable, and measure it.
How Do You Know Your RAG Index Is in Sync?
Measure it, because "the job ran" is not the same as "the index is right":
Documents skipped, changed, and deleted per run, plus embedding tokens spent.
Drift: active sources in the registry versus sources that exist at the origin.
Age of the oldest unprocessed change event.
A canary document that a scheduled job edits and then queries, asserting the new text is retrievable within your freshness target.
A retrieval debug endpoint makes the canary check easy to automate. Our guide to grounded RAG answers with citations in .NET walks through building one, and how to chunk documents for RAG in .NET covers the chunking decisions your pipeline version string should capture.
What to Do Next
Add the registry table and fingerprint check first, because it pays for itself on the next run. Switch replacement to write-new-then-delete-old second. Add the reconciliation sweep third, and treat the canary document as the test that proves all three keep working.
Frequently Asked Questions
How do I update a document in a vector store without re-indexing everything?
Store a content hash per source document in a registry, and only process documents whose hash changed. For a changed document, chunk and embed the new version, upsert the new chunks under keys that include the new hash, then delete the chunks carrying the old hash. Unchanged documents are skipped entirely, so each run costs embedding calls in proportion to what actually changed.
How do I delete old embeddings when a document is removed from a RAG knowledge base?
Reconcile the source against your registry on a schedule. List the document IDs that exist at the source, find registry entries that are active but missing from that list, delete their chunks with a filter on the source ID, and mark them deleted. Change events alone miss deletions, so the scheduled sweep is the safety net.
Does Microsoft.Extensions.DataIngestion support incremental ingestion?
Partly. Its VectorStoreWriter<T> has an IncrementalIngestion option, enabled by default, that deletes a document's previously stored chunks after writing the new ones. It does not detect whether a document changed, and it cannot remove chunks for documents that were deleted at the source, so you still need change detection and reconciliation around it. The library is still in preview.
What happens to my RAG index when I change the embedding model?
Every stored vector becomes incompatible with new query vectors, even when the dimension count is the same. Treat it as a full rebuild into a new collection: include the model name in your pipeline version so every fingerprint changes, backfill in the background, switch queries to the new collection, and delete the old one after verifying retrieval quality.
How often should a RAG index be re-synced?
Use change events for near real-time updates, plus a reconciliation sweep on a schedule that matches how quickly stale content becomes harmful. For many support knowledge bases an hourly or nightly sweep is enough. Anything removed for legal or safety reasons should bypass the schedule and be deleted as soon as the event arrives.
About the Author
I'm Celin Daniel, Co-founder of Coding Droplets. I've been building .NET and ASP.NET Core systems in production for 13+ years - APIs, distributed backends, enterprise platforms. Everything I write here comes from real shipping experience: patterns that held up, trade-offs that bit us, and lessons learned the hard way.
GitHub: codingdroplets
YouTube: Coding Droplets
Website: codingdroplets.com
