LLM Chat in .NET with IChatClient: The Complete Guide

April 9, 2026 9 minutes read

The IChatClient interface from the Microsoft.Extensions.AI package offers a clean, unified way to integrate LLMs like OpenAI, Copilot, or Anthropic into your application. It’s a solid abstraction—but the official docs only get you part of the way. We needed to go further before integrating it into our product. This guide is the result of that deep dive.

You’ll learn how to:

• Use IChatClient across multiple LLM providers
• Discover and select available models for each LLM provider
• Build both stateless and stateful chat experiences
• Expose your own C# methods to the LLM (tooling / function calling)
• JSON format the response
• Track token usage and access telemetry
• Cache responses (exact match and semantic match)

You don’t need to read this post linearly. Start with how the chatClient is built and how the chat loop works, then feel free to jump between sections in whatever order suits you.

When you’re ready to experiment, head straight to the final section — Putting It All Together — where you’ll find a ready-to-compile Program.cs and Project.csproj.

Defining LLM Providers with an Enumeration

Let’s start by defining the LLM providers we’ll work with using an enumeration. This could easily be extended to include a local Ollama model or an Azure OpenAI deployment, but for this guide we’ll focus on these three major providers:

Getting API Keys for Each LLM Provider

To use each LLM, you’ll need a token or key—here’s the step-by-step guide to obtain it:

Hardcoding secrets like API keys directly in source code is fine for quick experiments, but make sure to move them to environment variables before using this in anything serious.

Building the IChatClient object from the LLM provider choosen

Before instantiating ChatClient for any provider, add these packages to your .csproj:

The imported Anthropic package is the official version. There is also a non-official Anthropic.SDK package, which caused a version mismatch with Microsoft.Extensions.AI in our environment. Although it certainly works fine, here we are using the official Anthropic package.

Assuming you have the llmProvider and the model identifier as string, here is the code to instantiate the ChatClient object. Here, we use fully qualified names to avoid adding multiple using statements for the different providers.

The Chat Loop

Here is the chat loop code, which is fairly straightforward. Notice how we fetch the LLM response using successive chatClient!.GetStreamingResponseAsync() calls. For long responses, this allows your application to display the output as it arrives, improving responsiveness. Later, we’ll show how to use chatClient!.GetResponseAsync(...) to receive the entire response in one go.

Stateless vs. Stateful Clients

So far, our chat has been stateless, meaning each query-response loop operates independently. Often, however, you want a stateful chat implemented through a chatHistory list. The example below illustrates why:

Here is the updated chat loop code, now using a chatHistory list:

Chat Reducing (Experimental)

With a chatHistory list, messages accumulate, so each additional query consumes more tokens. This is why chat reduction is often applied: it helps manage conversation history by either limiting the number of messages or summarizing older ones once the conversation grows too long.

The Microsoft.Extensions.AI library includes an experimental SummarizingChatReducer, which automatically condenses older messages while preserving context. It can be used as shown, though we haven’t been able to get it working reliably yet.

We’ll update this post once it works.

IChatClient Dependency injection

IChatClient implementations are typically provided to an application through dependency injection (DI). The code below demonstrates how to inject the chatClient object and how to access it from other parts of the application:

This code requires this package:

Listing available models for a provider

We want to let the user choose from the most popular LLM providers and, for the selected provider, view and select from the available models.

The following code shows how to list available model IDs for each provider. The desired model ID is then passed when constructing the IChatClient object.

The JSON returned by each provider is not standardized. Below are excerpts of the responses from each provider:

• Github Copilot JSON:

• OpenAI JSON is formatted this way:

• Anthropic JSON is formatted this way:

RAG and Agentic Patterns in C#: Letting the LLM Call Your Functions

One powerful feature of IChatClient is the ability to define one or more C# functions that the LLM can call when building the response, if needed. This enables Retrieval-Augmented Generation (RAG), allowing the LLM to access specific data it wouldn’t otherwise know—similar to how tools work on an MCP server. For more details, see our post Developing an MCP Server with C#: A Complete Guide, which shares insights from building the open-source NDepend.MCP.Server.

Letting the LLM decide which function to call and with which arguments is a step toward truly agentic behavior.

To make your functions available, first import the following package:

Then, the code below defines the GetCurrentTime() function and registers it as a chat option. You can describe the function and its parameters with the [DescriptionAttribute]:

Notice the chatClient feature chaining pattern: chatClient = chatClient.AsBuilder().UseFunctionInvocation().Build(); We’ll see more examples of this chaining pattern later.

Finally, you need to pass the chat options into the chat loop like this: chatClient!.GetStreamingResponseAsync(chatHistory, options)

And there you have it! We can now debug calls made by the LLM to our GetCurrentTime() function when asking “what time is it”:

Chat Options

The ChatOptions class introduced earlier—used to define functions callable by AI—also exposes several additional configuration properties:

• public float? Temperature { get; set; }
  Controls the randomness of the model’s output. Lower values produce more deterministic and consistent responses, while higher values increase variability and creativity.
• public long? Seed { get; set; }
  Specifies a seed value that helps ensure reproducible results across multiple executions, provided the underlying service supports it.
• public Microsoft.Extensions.AI.ReasoningOptions? Reasoning { get; set; }
  Defines the reasoning configuration applied to the chat request, allowing fine-tuning of how the model approaches problem-solving.
• and more

JSON Response Formatting

The ChatOptions property public Microsoft.Extensions.AI.ChatResponseFormat? ResponseFormat { get; set; } can be used to format the response. For example:

This prints:

A JSON schema can be forced:

This prints:

Counting Token

The cost of using an LLM is measured in tokens, making it essential to track both input and output token usage. Our chat loop can be modified as shown below. Currently, we didn’t find a better way to access UsageContent than this code: response.Contents.OfType<UsageContent>().FirstOrDefault()

Due to the accumulation of chatHistory, we can observe how the number of input tokens increases with each query:

If we comment out the options to skip passing our GetCurrentTime() function—like this: GetStreamingResponseAsync(chatHistory/*, options*/)—we notice that far fewer input tokens are consumed (19 instead of 385!!). This illustrates a downside of AI functions (and MCP tools): they significantly increase token usage.

Show Telemetry

You might also be interested in LLM telemetry, which provides insights far beyond just token usage. OpenTelemetryChatClient acts as a wrapper around your standard IChatClient implementation. By using it, you gain automatic observability and telemetry for all interactions with your LLMs, without changing your existing code. Here’s why it’s useful:

1. Centralized Monitoring – Track requests, responses, and errors from all your LLM providers in one place.
2. Performance Metrics – Measure latency, throughput, and resource usage for each chat interaction.
3. Distributed Tracing – See the full journey of a request, especially if your system calls multiple services before or after the LLM.
4. Debugging Made Easy – Identify bottlenecks or failed calls quickly with structured logs and traces.
5. Non-Intrusive Integration – Works seamlessly with any IChatClient implementation, so you don’t have to modify your core logic.

To have access to telemetry just import this package…

… and then chain a TracerProvider with the chatClient:

Here is what telemetry on console looks like:

Receive the Full Response at Once Instead of Streaming

Until now, we’ve requested streaming chat responses, but it’s also possible to receive the full response at once. This simplifies the code, though the chat becomes less responsive for the user. Note that this approach does not affect token consumption:

Chat History System Prompt

You can shape the LLM’s behavior through the system prompt, which is the standard way to inject a persona or skills via IChatClient. Here’s how:

As the screenshot below illustrates, the system prompt counts toward your input token usage:

Response Caching — Exact Query Match

Tokens can be costly, and if thousands of users are asking similar questions, it’s worth caching LLM responses. First, we’ll look at caching responses based on an exact match of the query text. The next section will cover semantic matching, where queries like ‘Who is your LLM provider in 10 words?’ and ‘Who is your LLM provider in ten words?’ return the same response. Below is the chatClient chaining code for exact-match caching:

To compile this code import this package:

To detect when a response comes from the cache, we measure the time it takes to retrieve it. Responses from the LLM take a few seconds, while cached responses are returned instantly. We also disable chatHistory; otherwise, asking the same query a second time wouldn’t be treated as identical.

Here is a demonstration. Even a single-character difference causes the cache to miss, triggering a new LLM call. Additionally, when a function is invoked, the response cannot be cached.

Response Caching — Semantic Query Match

Let’s wrap up this blog post with semantic query matching. The idea is to represent each query as a high-dimensional vector, also called an embedding. When a new query comes in, we measure how close its embedding is to those of previously seen queries. This closeness — the cosine similarity — is a scalar between 0.0 and 1.0, where 1.0 means the vectors point in exactly the same direction. If the similarity exceeds a defined threshold, we consider the queries semantically equivalent and return the cached answer directly, sparing an expensive LLM call. Here is the kind of behavior we want:

The KnowledgeBase class

See below our KnowledgeBase class, which stores previous queries alongside their answers and embeddings. We use a similarity threshold of 0.8 — chosen arbitrarily for this example, but in practice this value deserves careful tuning. Set it too high and the cache rarely triggers, making it useless. Set it too low and it starts returning answers that don’t actually match the incoming query.

Note that the embedding client is wired to OpenAI.Embeddings.EmbeddingClient and Keys.OPENAI, but this is purely an implementation detail. The chatClient and the embedding client are completely independent, so the KnowledgeBase works regardless of which LLM powers the chat side.

If embeddings are persisted to a database, make sure to always generate them with the same embedding model — swapping it out will invalidate every stored vector.

The following NuGet package is required to compile this class:

The chat loop updated

Below see the updated chatLoop, now querying the knowledge base before falling back to the LLM.

For brevity, we stripped out chatHistory — but unlike the exact match approach from the previous section, history doesn’t interfere with semantic matching, which only looks at the raw userInput string.

That said, caching across different history contexts should be used with caution: if the same query is asked twice but the conversation has taken a different path, the cached answer may no longer be appropriate.

For larger caches, the KnowledgeBase class could be extended to index the vector column, reducing similarity lookup to constant time regardless of how many embeddings are stored. We’ll leave that as an exercise — your preferred LLM would be happy to walk you through the implementation if you ever need it.

As a fun aside, the embeddings we worked with during testing came out at 1,536 dimensions each.

Putting It All Together

To finish, here is the ready-to-compile Program.cs and Project.csproj demonstrating the multiple LLM provider and their models support, the AI function, and the stateless chat loop with query embedding cache. Just drop in your API keys before running.

The C# code:

Conclusion

Demystifying IChatClient was a rewarding exercise, and one with a practical motivation — we have ideas for bringing it into NDepend to make the tool smarter.

We also evaluated the GitHub Copilot SDK, which imposes a higher barrier to entry — users must have the GitHub Copilot CLI and a valid key — but offers stronger potential for agentic workflows, making it a compelling direction for future iterations.

But keep in mind that the interface itself is just plumbing. The real value lies in your prompt design and the AI-powered logic you build around it, which is what ultimately guides the LLM toward something genuinely useful for your users.