Querying Reliable AI Resources with Telerik Agent Tools API

Telerik Document Processing Libraries not only let you create RAG resources for your LLMs. They let you integrate those libraries into Microsoft’s latest agent-based tools.

In my previous post, I showed how to use Progress Telerik Agent Tools API to create a workflow that loads content into a Resource-Augmented Generation (RAG) resource that can be used with Large Language Models (LLMs). That RAG resource, integrated with an LLM in your application, helps your users get reliable, grounded answers, driven by the content you load into your resource.

This post shows how to tie your RAG resource to an LLM and integrate the combination in an application that allows users to query the resource—or any combination of resources that you’ve created. Specifically, I’ll show how to use Microsoft’s current technology for querying an AI resource, the ChatClientAgent object.

Telerik Agent Tools API provides a collection of toolsets that the chat client agent works with to query your RAG resources. You just have to load your RAG resource(s) into in-memory repositories, attach the appropriate toolsets and pass the resulting tools to a chat client agent. Once you’ve done that, you can submit prompts to the agent and get back the results driven by the RAG resources you’ve loaded.

Configuring Your Project

To create an application that can query your resources, you first need the Telerik.Documents.AI.* NuGet packages that work with the documents in your RAG resource (I covered those libraries in my previous post).

After that, you also need two Microsoft AI NuGet packages:

• Azure.AI.OpenAI
• Microsoft.Agents.AI.OpenAI

I used an ASP.NET Web API project for my case study so I also needed to add the Microsoft.AspNetCore.OpenApi package.

As I write this, the Microsoft.Agents.AI.OpenAI package was in “release candidate” mode which means that, while its interfaces and functionality are fixed, I needed to use the “prerelease” option when adding the package. Having said that, by the time you read this, the package may have moved to “latest stable” status (this is a very agile environment).

Configuring a Chat Client Agent

Before you create your ChatClientAgent agent object, you need to assemble a set of tools, tied to one or more of your RAG resources. Your first step is to create a repository of the right type (PDF, spreadsheet, Word/Word-related) and load the file that holds your RAG resource into that repository.

For my case study, I’m only working with PDF documents—what Progress Telerik calls “Fixed” documents—so I created a IFixedDocumentRepository pdfRepo repository (all the repositories share a common interface so they all look very much alike).

Once I created the repository, I loaded my RAG resource using the repository’s Import method. The Import method must be passed a FileStream object pointing to the resource and its format which, in this case, was DocumentFormat.PDF (see my previous post for the details on creating that resource).

Here’s the code that loads my case study’s repository:

IFixedDocumentRepository pdfRepo = 
               new InMemoryFixedDocumentRepository( new TimeSpan(0, 2, 0) );
if ( Path.Exists(repoPath) )
{
    string name =pdfRepo.Import(
                                        FileStream(repoFilePath, FileMode.Open, FileAccess.Read),
                                       DocumentFormat.PDF,
		         Path.GetFileNameWithoutExtension(repoFilePath)
                                     );
}

If you provide the name of the file as the third parameter, the Import method returns that name (and returns “ImportedDocument” if you don’t provide the third parameter).

To support having the chat client agent query my PDF repository, I just need the FixedDocumentContentAgentTools toolset. To create that toolset, I have to pass two parameters: the repository itself and a folder that holds any images used in those PDF documents. When attaching a toolset to a repository, you’ll always have to pass the repository parameter, but other toolsets will require different parameters (and, often, no other parameters).

Once I’ve created that toolset, I extract its tools using the toolset’s GetTools method and add those tools to a list of AITool objects that will, eventually, be passed to my chat client agent. The code to do that looks like this:

pdfReadTools = new FixedDocumentContentAgentTools(pdfRepo, repoPath);
List<AITool> queryTools = new List<AITool>( pdfReadTools.GetTools() );

If I wanted to add more tools to my list, I would use the list’s AddRange method. As an example, the following code:

• Creates a repository for holding spreadsheets
• Imports a RAG resource file of spreadsheets into that repository using the repository’s Import method
• Attaches the SpreadProcessingReadAgentTools toolset (which only needs to be passed a reference to the repository)
• Extracts the toolset’s tools and adds them to my list of AITools

IWorkbookRepository workbookRepo = 
              new InMemoryWorkbookRepository( new TimeSpan(0, 2, 0) );
workbookRepo.Import(workbookRepoPath, DocumentFormat.XLSX);

SpreadProcessingReadAgentTools workbookTools = new(workbookRepo);

queryTools.AddRange( workbookTools.GetTools() );

Querying Your Repositories

Now that I have a list of tools, I’m ready to create an agent. There are three steps to doing that (but you can do it in one line of code).

First, you need to create a AzureOpenAIClient object which, in its simplest form, just requires passing the URL for the LLM deployment you’ve created and the authorization key for that deployment.

One note: Using an authorization key is probably fine for development but, in production, you should be authorizing access using something more robust (e.g., Managed Identities in Entra ID). If you’re using an authentication key, then you shouldn’t hardcode into your application as I do here but move that key to some more secure location (e.g. An Azure Key Vault).

Once you’ve created your AzureOpenAIClient object, your second step is to call its GetChatCient method to configure and return a ChatClient object. The GetChatClient method just needs to be passed the name of the deployment you created for your LLM.

Finally, you need to call your ChatClient object’s AsAIAgent method to configure your agent. You can pass a variety of parameters as part of configuring your agent. I settled for specifying a name for my agent, some instructions on how my agent is to answer questions, and my list of tools:

AIAgent agent = new AzureOpenAIClient(
                                      new Uri(url),
                                      new AzureKeyCredential(apiKey))
        .GetChatClient(deploymentName)
        .AsAIAgent(
                instructions: "You provide guidance to Azure software developers",
                name: "Async App Expert",
                tools: queryTools);

With your chat client agent in hand, you process your user’s prompts by calling the agent’s RunAsync method and passing the prompt. The agent will return an AgentResponse object which has a Messages collection holding a list of responses drawn from the repositories associated with your tools (you probably want the first message). The Text property on a message will give you the agent’s response.

Typical code would look like this:

AgentResponse response = await agent.RunAsync("What do NuGet packages do I need”);
string responseText = response.Messages[0].Text;

Interacting with Your RAG-Enabled Sources

Of course, you’ll also want to create a UI for your users to interact with when querying your RAG resource. In earlier posts, I showed how to leverage Telerik tools to create dedicated user-friendly UIs in Blazor or JavaScript. Alternatively, instead of creating a UI dedicated to your RAG-enabled resource, you might want to more tightly integrate your resource into a JavaScript application’s UI. 

But, really, it’s up to you how you’ll use your RAG resource to support your users.