Create a custom RAG-enabled AI agent for an LLM with Progress Telerik Document Processing Libraries AI connectors.
The problem with general-purpose AI agents is that they are subject to “hallucinations” (these days, we say that those agents produce answers that aren’t “grounded”).
One solution to that problem is to create a custom AI agent with your own content and have your agent use that content to generate grounded answers (this is referred to as Retrieval Augmented Generation or RAG). Effectively, you can create a dedicated, reliable AI assistant for your users for any aspect of your organization.
Progress Telerik Document Processing Libraries (DPL) include a set of AI connectors that support creating those custom RAG-enabled AI agents of your own using any of the existing Large Language Models (LLMs) available to you in Azure or on your own computer/server using Ollama. And, once that agent is created, Telerik UI tools will let you integrate it into any application.
So, this post kicks off a series of posts where I’m going to walk you through all of the steps you need to create a custom AI agent by using Telerik DPL (and related AI tools), integrated with an LLM available on Azure or in Ollama. Once you’ve created an agent, I’ll show you how to add that custom agent into your application’s UI using AI-enabled Telerik UI components in either JavaScript or Blazor (though you could also create your frontend in ASP.NET, Windows Forms or .NET MAUI).
But … having said that, if all you want to do is add an AI prompt to your application, then, thanks to Telerik tools, this post might be the only one you need. At the end of this post, I’ll show you how to create a simple frontend for your application to use your LLM. In fact, if you’ve already got an LLM deployed (which is what this post is about), you might want to skip the rest of this post and go directly to that last section.
If you stick with Microsoft’s offerings, you actually have three options for creating the LLM your application will use. The good news here is that, thanks to Microsoft and Telerik tools, it doesn’t make much difference to your code which of those three options you use.
Your three choices are:
Microsoft Foundry: This gives you access to a wide variety of LLMs from multiple providers (including the OpenAI LLMs), while giving you integration with Azure security. Microsoft Foundry also gives you extensive monitoring for your LLM, enables data privacy and supports integration with other Azure services. This is Microsoft’s recommended option. I’ll call this the “Foundry option.”
Azure OpenAI: This is the easiest option to set up but also provides the least access to Azure functionality. The Azure OpenAI option gives you access only to OpenAI’s LLMs and only limited integration with other Azure resources (principally, security). With this choice you do, however, get access to the latest OpenAI LLMs, potentially before they show up in Microsoft Foundry. I’ll call this the “OpenAI-only option.”
Ollama: Unlike OpenAI or Microsoft Foundry, Ollama runs on your local computer, rather than in the cloud. Since there are no API fees, that makes Ollama an attractive choice for cheaper solutions (provided you have a powerful enough development computer). I’ll call this the “Client-side option.”
While I’m treating the OpenAI-only option as a separate case from the Foundry option, the OpenAI-only option is actually hosted in Microsoft Foundry. As a result, the processes for setting up the OpenAI-only or the Foundry option are very similar:
To start, surf to the Azure Portal and search for Microsoft Foundry. On the Microsoft Foundry page, in the menu on the left, expand the Use with Foundry node and click on either:
Either choice will open a page on the right. On that new page, click on the +Create menu choice at the left end of the menu running across the top of the page. That will start the wizard that will create your Azure resource.
If you opened this page from the Foundry node, you’ll start the wizard for creating your Foundry resource.
If you opened this page from the OpenAI node, you’ll get a dropdown list with two choices: OpenAI and Microsoft Foundry. This is, effectively, your last chance to switch to the Foundry option instead of OpenAI-only. Pick the OpenAI choice to open the wizard for creating an OpenAI-only resource.
Regardless of which wizard you’ve invoked, on the wizard’s first page you’ll need to provide a resource group, a name for your resource and a region for your LLM to run in. In addition, on the first page:
After that, it’s just a matter of “Next”ing your way through the rest of the wizard’s tabs.
Both the Foundry and OpenAI-only wizards include a Network tab to control access to your resource. You have three choices on the Network tab:
You would use the last two options if you were going to run your application on a VM on a virtual network or in an App Service. You don’t need to use the network choices to control access to your resource. You could control access to your LLM just through the resource’s Access Control/IAM roles, for example, or your application’s built-in security. However, for a production application, you should always lean toward more control rather than less and pick whichever of these options that makes sense for limiting network access to your resource.
The Foundry wizard includes three additional tabs that the OpenAI wizard does not:
For a proof-of-concept project, you can take the defaults on all these tabs.
Both wizards finish with the standard Tags tab and Review-and-create tab. When you get to that last tab, click the Create button to create your resource.
Your application interacts with a deployed instance of some LLM so your next step is to create a deployment that uses one of the LLMs available to your resource.
To do that, navigate to your resource and, on the Overview page, click on the Go to Foundry portal link at the top of the page. This will open a new tab in your browser displaying the ai.azure.com portal. This is where you will select the LLM you’ll use in your deployment.
To create a deployment, on the Microsoft Foundry page for your resource, in the menu on the left, click on the Model catalog node. That will take you to a page where you can select your LLM.
A warning: Over and above whatever criteria you have for getting the right model for your application, selecting a model may require some trial-and-error poking around. That’s because:
Once you open the page listing available LLMs, if you got to the page from a:
When you select an LLM, Azure will display a page with high-level information about your LLM. For my case study, I created three deployments, one each for these three LLMs:
To create your deployment, at the top of the page, click the button labeled “Use this model.” That will display a dialog where you can customize the LLM you’ve chosen, including giving your deployment a name (your deployment’s name will default to the name of your LLM). Click on the button at the bottom of the page labeled “Create resource and deploy” … and then be patient because this can take a moment.
After your deployment is created with your selected LLM:
Once you’re on your deployment’s page, you need three required pieces of information from the page to use in your application and one optional piece:
To return to your resource or your model, first surf to ai.azure.com and sign in. This portal is, currently, in the process of migrating to a new UI, so:
You have more to do to create your own custom RAG-enabled AI agent (load your own content, for example). But if all you want is an application that integrates an AI prompt, you’ve done everything you need to support that. You just need to create a frontend that lets the user query your LLM’s deployment. The Telerik AI Prompt will let you do that with minimum effort. I’ll cover that after showing how to set up the client-side option with Ollama.
To implement the client-side option, you first need to download Ollama and install it. To confirm that Ollama installed successfully, open a command window and enter ollama. You should get back a list of Ollama commands.
You next need to add the LLM you want to use to Ollama. To do that, open a command window and use Ollama’s pull command to install one of the LLMs from the Ollama registry. This command installs OpenAI’s gpt-oss LLM:
ollama pull gpt-oss
After using the pull command, you can use Ollama’s list command (which displays the installed LLMs) to confirm your installation.
The Ollama server should start automatically after installation (and after every reboot) and should be listening on port 11434 on localhost. You can confirm that by using Ollama’s serve command. If server isn’t running, it will start and report the address the server is listening on. If the server is already running, you’ll get an error message that includes address that Ollama is listening on.
Make a note of the address (probably http://localhost:11434) and the name of the LLM you installed—you’ll need both in your code.
I’m building up to a creating a custom AI agent over this series of posts. But, at this point, you’ve done everything you need to create an application with an AI prompt—it’s just a matter of adding a frontend to your LLM.
In my next posts, I’ll use Telerik Document Processing Library tools to create applications that allow users to apply AI to documents. However, once you’ve created an LLM deployment, you can tie that deployment into your application’s UI with a few lines of code in your application’s Program.cs file and then query your LLM through the Telerik AI Prompt component in your application’s UI.
To do that in a .NET 8 or later application, you first need to add the Microsoft.Extensions.AI.OpenAI NuGet package to your application. After that, you need to tie your model into your application by adding the following code to your application’s Program.cs file, inserting the information about your LLM that you gathered from your deployment’s information page.
For either of my Foundry or OpenAI-only options that code looks like this:
builder.Services.AddSingleton(new AzureOpenAIClient(
new Uri(“<url>”),
new AzureKeyCredential(“<key>”)
));
builder.Services.AddChatClient(services =>
services.GetRequiredService<AzureOpenAIClient>()
.GetChatClient(<deployment name>).AsIChatClient());
For Ollama, in addition to adding the Microsoft.Extensions.AI.OpenAI NuGet package, you’ll also need to add the OllamaSharp and Microsoft.SemanticKernel.Connectors.Ollama packages. After that, this single line of code will integrate Ollama into your application:
builder.Services.AddSingleton<IChatClient>(
sp => new OllamaApiClient("<Ollama address>", "<model name>")
);
Your last step is to add the Telerik AIPrompt to your application’s UI (available in JavaScript for Kendo UI, Blazor, ASP.NET AJAX, WinForms and .NET MAUI). That can be as simple as this in an application that supports Razor (e.g., ASP.NET or Blazor):
<TelerikAIPrompt/>
And that would give you this UI:
In a JavaScript application, you’ll need to add a <div> element, set the element’s id attribute, and write a couple lines of JavaScript code. I’ll cover that in a later post in this series.
You can now start your application, enter a prompt into the AIPrompt component (e.g., “Tell me about RAG-enabled AI applications”) and get a response.
There’s obviously far more that you can do with the AI Prompt … and I’ll return to that after my next post. Before that, however, I’m going to roll in Telerik Document Processing Library to let you load the content for your custom AI agent.
Explore Telerik Document Processing Libraries, plus component libraries, reporting and more with a free trial of the Telerik DevCraft bundle:
Peter Vogel is both the author of the Coding Azure series and the instructor for Coding Azure in the Classroom. Peter’s company provides full-stack development from UX design through object modeling to database design. Peter holds multiple certifications in Azure administration, architecture, development and security and is a Microsoft Certified Trainer.