# Maximum Context Length Exceeded in .NET AI APIs: Causes and Fixes

The chat endpoint works fine in testing. It works fine for the first dozen messages in a real conversation. Then a user who has been going back and forth all afternoon sends one more message and gets a 500, with this buried in the logs:

```text
This model's maximum context length is 128000 tokens. However, you requested
131204 tokens (127204 in the messages, 4000 in the completion).
Please reduce the length of the messages or completion.
```

The **maximum context length exceeded** error is a hard provider-side rejection, returned as HTTP 400 with an error code of `context_length_exceeded`. It is not transient, so retrying is pointless - a retry policy will simply burn three attempts and fail identically. In production I've seen this take down an AI support endpoint at exactly the worst moment, because the conversations that hit it first are the long, high-value ones from your most engaged users.

This article covers what the numbers in that message actually mean, the six causes I keep finding behind it, and the fix for each. If you want the working token budgeter and history trimmer rather than the pieces, the complete implementation is on [Patreon](https://www.patreon.com/CodingDroplets) with the tests that pin the edge cases down.

The durable fix is not a bigger model. It is treating the context window as a budget you spend deliberately. [Chapter 4 of AI-Powered .NET APIs](https://aiapis.codingdroplets.com/) covers exactly that - prompt templates as versioned assets, database-backed multi-turn conversation state, trimming strategies, and counting tokens before you send - inside one running ASP.NET Core support API.

[![AI-Powered .NET APIs](https://newsletter.codingdroplets.com/images/ai-api-course-banner-1.jpg align="center")](https://aiapis.codingdroplets.com/)

Targets .NET 10 with `Microsoft.Extensions.AI` 10.x.

## What the Error Actually Means

Every model has a fixed context window shared by **everything in the request and everything it will generate**. The message breaks the total down for you, and reading it properly tells you which fix you need:

*   `maximum context length is 128000` - the model's total window.
    
*   `127204 in the messages` - your input: system prompt, full conversation history, retrieved RAG context, tool definitions, and any images.
    
*   `4000 in the completion` - the space reserved for the answer, taken from your `MaxOutputTokens` setting.
    

The sum of the last two must be under the first. This is the detail most people miss: **the reservation for the completion counts against the window before the model generates a single token**. You can be comfortably under the limit on input alone and still get rejected because you asked to reserve 16,000 tokens for the reply.

In a .NET app the failure surfaces as a provider exception bubbling up through `IChatClient` - a `ClientResultException` from the OpenAI client, a `RequestFailedException` from Azure - carrying the 400 status. Because `Microsoft.Extensions.AI` is a thin abstraction, the provider's message reaches you intact, which is useful: the token breakdown in it is the fastest diagnostic you have.

## Cause 1: Unbounded Conversation History

This is the cause in the large majority of real incidents. Each turn appends a user message and an assistant message to the history, the whole history goes back on every call, and nothing ever removes anything. Growth is linear in turns and the failure is guaranteed, just deferred.

The tell is that it works for short sessions and fails for long ones. If you can reproduce it by having a conversation for twenty minutes, this is your cause.

**The fix** is a trimming policy applied before every send. The policy has three non-negotiable rules:

1.  **Always keep the system message.** Dropping it is worse than the error - the model silently loses its instructions and starts behaving differently, and nobody notices for weeks.
    
2.  **Drop from the oldest end**, never the newest. Recent turns carry the conversational state that matters.
    
3.  **Drop message pairs**, not individual messages. Leaving an orphaned assistant reply with no preceding user turn confuses the model and wastes tokens.
    

Where the older context genuinely matters, summarise instead of dropping: collapse the oldest turns into a single compact summary message and keep it pinned behind the system prompt. That costs one extra model call per compaction, so trigger it on a threshold rather than every turn. Our guide on [managing LLM conversation history in .NET](https://codingdroplets.com/llm-conversation-history-dotnet) covers the storage side.

## Cause 2: No Reserve for the Completion

You trim the input to just under the window and the request still fails, which feels like a bug in your arithmetic. It is not - you forgot the completion.

**The fix** is to budget explicitly and subtract the reservation up front:

```csharp
const int Window = 128_000;
const int Reserve = 4_000;                 // must match ChatOptions.MaxOutputTokens
const int Safety  = 500;                   // template overhead, role markers, drift

int inputBudget = Window - Reserve - Safety;
```

Then set `MaxOutputTokens` on `ChatOptions` to the same `Reserve` value you budgeted for. If those two numbers disagree, the provider uses its number and your arithmetic is decorative.

The safety margin is not superstition. Token counts differ slightly between your local tokenizer and the provider's accounting for message role markers and formatting overhead. Leaving a few hundred tokens of headroom converts a hard 400 into a non-event.

## Cause 3: RAG Context Stuffing

An endpoint that retrieves the top ten chunks and pastes them all into the prompt has handed control of its token budget to whatever the ingestion pipeline produced. One badly chunked source document - a 40-page PDF that became a single chunk - blows the window on its own.

**The fix** is to cap retrieval by **tokens**, not by chunk count. Add chunks in relevance order until the context budget is spent, then stop. This is strictly better than a fixed top-k because it adapts to chunk size automatically.

Cap chunk size at ingestion too. If any single chunk can exceed a meaningful fraction of your window, your chunking strategy needs work, and [chunking documents for RAG in .NET](https://codingdroplets.com/chunking-documents-rag-dotnet) covers the sizes that hold up.

Passing fewer, better chunks usually improves answer quality as well - more context is not better context.

## Cause 4: Tool Definitions Eating the Window

Every tool you expose is serialised into the request as a JSON schema with names, descriptions and parameter types. Thirty tools with thorough descriptions can consume several thousand tokens **before your user has typed anything**.

The tell is that the error appears on short conversations, which rules out history growth. Log the token count of the request with an empty message list; if that number is large, this is your cause.

**The fix** is to expose fewer tools per call. Filter the tool set by intent or by the current step of the workflow rather than registering everything globally. This also improves tool-selection accuracy, since a model choosing among five relevant tools makes better decisions than one scanning thirty. Our post on [securing LLM tool calling](https://codingdroplets.com/securing-llm-tool-calling-aspnet-core) covers the allow-listing patterns that give you this for free.

Trim the descriptions too. They need to be precise, not prose.

## Cause 5: A Single Oversized User Input

Someone pastes a 200-page contract into a summarise endpoint. No history involved, no RAG, just one message larger than the window.

**The fix** has two parts. First, **reject early**. Count tokens at the API boundary and return a 413 with a clear message rather than letting the request travel to the provider and fail there. You get a better error, a faster response, and no wasted spend.

```csharp
var tokenizer = TiktokenTokenizer.CreateForModel("gpt-4o");
int tokens = tokenizer.CountTokens(request.Text);
if (tokens > inputBudget)
    return Results.Problem(statusCode: 413,
        detail: $"Input is {tokens} tokens; the limit is {inputBudget}.");
```

`TiktokenTokenizer` lives in the `Microsoft.ML.Tokenizers` package and gives you the same BPE counting the provider uses, locally and for free. It also exposes `GetIndexByTokenCountFromEnd`, which is what you want when trimming a string to a token budget rather than guessing with character counts.

Second, for inputs that legitimately exceed the window, **split and combine**: process the document in sections and merge the results. That is a design decision, not an error path, and it belongs in the endpoint from day one if long documents are in scope.

## Cause 6: A Model Swap Nobody Costed

The config changed from a 128k model to a cheaper 32k one, or a fallback provider kicked in with a smaller window, and prompts that fit yesterday do not fit today. This one is nasty because the change looks unrelated to the failure.

**The fix** is to make the window a property of the configured model rather than a constant compiled into your prompt builder, and to fail loudly at startup if the configured window is smaller than your budget arithmetic assumes. If you route across model tiers - and [model routing and tiering in .NET](https://codingdroplets.com/llm-model-routing-tiering-dotnet) is a good idea for cost reasons - the budget must be recomputed per route, not once globally.

## How Do You Prevent This Instead of Fixing It?

Four controls, in order of value:

*   **Count before you send.** A token budgeter in front of every model call turns an unpredictable provider 400 into a deterministic decision you control.
    
*   **Record usage on every response.** `ChatResponse.Usage` exposes `InputTokenCount`, `OutputTokenCount` and `TotalTokenCount`. Emit them as metrics and you will see the ceiling approaching days before anyone hits it. Wiring that into traces is covered in [OpenTelemetry for AI endpoints](https://codingdroplets.com/opentelemetry-ai-endpoints-aspnet-core).
    
*   **Alert on the ratio, not the failure.** An alert on "requests above 80 percent of the window" is an early warning. An alert on the 400 is an incident report.
    
*   **Test with a long conversation.** A test that replays fifty turns and asserts the request still fits catches every regression in this article. Most AI test suites only ever exercise turn one.
    

## Frequently Asked Questions

### Should I Retry a Maximum Context Length Exceeded Error?

No. It is a deterministic HTTP 400 - the identical request will fail identically every time. Exclude `context_length_exceeded` from your retry policy explicitly, otherwise your resilience pipeline turns one failure into three, tripling latency for no benefit. Retries belong on 429s and 5xx responses, as covered in [resilient LLM calls in .NET](https://codingdroplets.com/resilient-llm-calls-dotnet).

### How Do I Count Tokens in C# Before Calling the Model?

Use the `Microsoft.ML.Tokenizers` package. `TiktokenTokenizer.CreateForModel("gpt-4o")` gives you a tokenizer matching the model's encoding, and `CountTokens(text)` returns the count locally with no network call. Add a small safety margin, because the provider also counts role markers and message formatting overhead that a raw string count does not capture.

### Does Trimming History Make the Assistant Forget Things?

Yes, and that is the trade-off you are choosing. Dropping old turns loses whatever information they held. Where that matters, summarise the oldest turns into one compact message instead of deleting them - it preserves the substance at a fraction of the tokens, at the cost of one extra model call when compaction triggers.

### Will a Model With a Bigger Context Window Solve This?

It moves the ceiling, it does not remove it. Unbounded history growth will exhaust a one-million-token window too, just later and far more expensively, since you pay for every input token on every turn. A larger window buys you room to implement a budget properly; it is not the budget.

### Why Does the Error Mention Tokens I Did Not Send?

Because the count includes everything: your system prompt, the full conversation history, retrieved context, serialised tool schemas, images, and the completion reservation from `MaxOutputTokens`. Log the token count of each component separately once, and it is usually immediately obvious which one is the problem.

### Where Is the Official Guidance on Token Limits?

Each provider documents its own per-model windows and error codes, and those are the numbers to trust. On the .NET side, the [Microsoft.Extensions.AI documentation](https://learn.microsoft.com/en-us/dotnet/ai/microsoft-extensions-ai) is the reference for `ChatOptions`, `UsageDetails` and how provider errors surface through `IChatClient`.

* * *

## About the Author

I'm Celin Daniel, Co-founder of [Coding Droplets](https://codingdroplets.com/). I've been building .NET and ASP.NET Core systems in production for 13+ years - APIs, distributed backends, enterprise platforms. Everything I write here comes from real shipping experience: patterns that held up, trade-offs that bit us, and lessons learned the hard way.

*   GitHub: [codingdroplets](http://github.com/codingdroplets/)
    
*   YouTube: [Coding Droplets](https://www.youtube.com/@CodingDroplets)
    
*   Website: [codingdroplets.com](https://codingdroplets.com/)