Coding Azure 14: Accessing Key Vault Secrets from a Server-Side Application (and Testing with Secret Manager)

You have three choices for retrieving secrets from an Azure Key Vault: from your environment (simple), for your appsettings file (compatible) and from code (flexible). Here’s how to implement all three.

You shouldn’t be putting confidential information (database passwords, credentials for databases and App Registrations, etc.) in your code where it can be seen by anyone who has access to your code—and that includes you. The right answer is to put those secrets in an Azure Key Vault and then securing that vault so that only your application can read values from the vault.

In this post, I’m going to cover all your options for accessing the Key Vault from server-side code and, also, how to test that code from either Visual Studio or Visual Studio Code using Secret Manager.

The Problem and the Solutions

Currently, the code for my case study’s Web Service includes the id of the Managed Identity that gives my application permission to read tables in my Azure SQL database—essentially, that Managed Identity forms the credentials for accessing data in my database. I want to put that value in a Key Vault so that my application has only the code to retrieve that value into a variable (I’m going to call that variable DbIdentityId). I can then incorporate that variable into my database’s connection string like this:

SqlConnection con =
    new SqlConnection($@"Server=tcp:warehousedbserver.database.windows.net;
                        User Id={DbIdentityId};
                        Initial Catalog=WarehouseDB;
                        Authentication='Active Directory Managed Identity';");

It gets worse. My case study’s server-side frontend claims an App Registration that gives my frontend permission to call my Web Service. The credentials for claiming that App Registration are sitting in plain text in my application’s appsettings.json file, like this:

"AzureAd": {
   "ClientSecret": "Hp68Q~4RSZWZAK5OV195jxCbi1i4oZTUIx6kva57",
   "Instance": "https://login.microsoftonline.com/",
   "Domain": "warehousemgmtfrontendaspnet.azurewebsites.net",
   "TenantId": "e98e6863-ebf3-4b73-bd25-e2826509a461",
   "ClientId": "80e0a00e-8cf2-4cfe-8a46-cc1dec17bf3b",
   "CallbackPath": "/signin-oidc"
 }

That ClientSecret property is the key part of the credentials that enable my application to claim the App Registration. I also want to move out of my appsettings.json file and into the Key Vault, ideally without disrupting the code that uses that property.

Fortunately, with a server-side frontend, you have three options for accessing values in your vault. You can:

• Retrieve Key Vault values into your App Service’s environment variables and have your application code read the values (simplest method)
• Write code to read the values from your App Service (the most flexible method)
• Integrate your appsettings.json file with your Key Vault (the method that supports processes that read from the appsettings.json file)

Accessing Secrets with Environment Variables

I could just put my Managed Identity id into my App Service’s Environment variables. That would limit the value visibility to those who have administration access to my App Service (which might not even be me). I can go one step further, however: If my Managed Identity id is stored in a Key Vault, I can create an Environment variable that holds just a reference to the key in the vault. By doing that, only vault administrators will be able to see the value in the vault and only applications running in my App Service will be able to read the value.

But beware: When your application’s code reads an Environment variable, it isn’t reading from the Key Vault. Your Environment variable holds a value from the Key Vault that can be as much as 24 hours old (or from whenever the App Service was last restarted, whichever is sooner). However, my Managed Identity id isn’t likely to change so it’s a good option for an Environment variable.

If, however, your Key Vault value is something that you cycle regularly (for example, the client secret for claiming an App Registration), you will want to verify that your App Service’s Environment variables are also up-to-date. To do that, after cycling your value in the Key Vault, return to your App Service’s Environment variables page and use the Pull reference values option on that page to refresh all your Environment values. Refreshing your values will also restart your App service, so pick the right time to do this.

Creating an Environment Variable

To use your App Service’s Environment variables, first surf to your App Service in the Azure Portal and, in the menu on the left, expand the Settings node and select Environment variables. Then, in the App Settings tab, click on the + Add option at the left end of menu beside the Search textbox.

The panel that opens on the right lets you define your Environment variable. At the top of the panel, give your variable a meaningful name. For my case study, I’m storing the id of the Managed Identity that gives my Web Service access to my Azure SQL database so I gave the Environment variable the name DbIdentity.

Below that, where you’d normally put in the value for your environment variable, put in the function that will retrieve that value from the key vault: the Microsoft.KeyVault function, passing the function’s name parameters:

• The name of your vault
• The name of the key for the value in the vault

Since I called my Key Vault WarehouseMgmtVault and gave the key holding my Managed Identity’s id the name DbAccessIdentity, I set my Environment variable’s value to the following:

@Microsoft.KeyVault(VaultName=WarehouseMgmtVault;SecretName=DbAccessIdentity)

That @ symbol at the start of the function name is important—it flags to the App Service that you’re calling a function. If you omit it, your Environment variable is filled with the string Microsoft.KeyVault(… rather than the value from the vault. Don’t ask how I know this.

Supplying just the vault and key names will give you the value from the latest version of the key. (Remember: you never replace a key in a Key Vault when you give a key a new value; you always create a new version of the key.) If you want to always pick up a specific version of the key, you can add a SecretVersion parameter to the call.

The panel that lets you create an Environment variable also includes a checkbox labeled Deployment slot which I’ll discuss at the end of this series as part of deploying your application.

With all your parameters entered, click the Apply button at the bottom of the panel to close the panel and return to your App Service’s list of Environment variables. Because changing Environment variables causes your App Service to shut down and restart, potentially disrupting anyone actually using the App Service, you’ll be asked to confirm your change.

When the list of Environment variables redisplays, you may get a red X in the list’s Source column and a link labeled “Key Vault”—that indicates that there’s a problem with your KeyVault function. Clicking the link will give you an error message describing the problem so that you can fix the problem. When you do fix your problem, that red X won’t go away until you click the Apply button at the bottom of the page.

You can test your Environment variables by clicking the Pull reference values menu choice at the right end of the menu across the top of page. That will load the variable with the current value of the secret and, as with changing or adding an Environment variable, will also restart your service.

Retrieving an Environment Variable

Now you’re ready to retrieve your Environment variable in an ASP.NET application. You can do that explicitly by calling the static GetEnvironmentVariable method of the Environment class. This code works in your Program.cs file, in a Razor page, or in a controller class:

string? DbIdentityId = Environment.GetEnvironmentVariable("DbIdentity");        

However, by default, your application’s IConfiguration object automatically searches your Environment variables so you don’t need to use the Environment object to retrieve an Environment variable. This code in your Program.cs file will work just as well (and, as you’ll see later on, can simplify your testing).

string? DbIdentityId = app.Configuration["DbAccessidentity"];;        

In a Controller class, you’ll need to retrieve the IConfiguration object in your class’s constructor and then use it:

  private readonly IConfiguration config;
  public Products(IConfiguration config)
  {    
      this.config = config;
  }
//…later
string? DbIdentityId = config["DbAccessidentity"];;

While using your IConfiguration object simplifies testing, using the Environment class in your code does make it more obvious to the next programmer where your secret value (or the reference to it) is stored.

Accessing Client Secrets with Code

If using Environment variables doesn’t give you the flexibility your application needs, you can add some code to your application to retrieve the value. You might want to do this if, for example, you want to verify that you always have the latest value from the vault.

To retrieve a secret, you need to create a SecretClient object, passing two parameters when you do:

• A Uri object loaded with the URL for your Key Vault (you can get that URL from your vault’s Overview page)
• A DefaultAzureCredential which will pick up the system-assigned Managed Identity that you added to the App Service

Once you’ve created the SecretClient, you can use its GetSecret method, passing the name of your key. The GetSecret method returns a KeyVaultSecret object which has a Value property to hold your secret’s value.

Typical code to retrieve the DbIdentity key in my WarehouseMgmtVault Key Vault would look like this for:

Uri kvu = new Uri("https://WarehouseMgmtVault.vault.azure.net/");
SecretClient sc = new SecretClient(kvu,
                                                                            new DefaultAzureCredential()
                                                                           );
KeyVaultSecret kvs = sc.GetSecret("DbIdentity"); ;
DbIdentityId = kvs.Value;

It would probably be a good idea to keep your vault’s URL in your appsettings.json file so that you can change the URL as you move your app from development to test to production.

Accessing Key Vault from appsettings.json

You can also use the Key Vault to hold secrets that you are retrieving from your appsettings.json file using the ASP.NET IConfiguration object.

By default, in an ASP.NET app, the IConfiguration object retrieves settings from the application’s appsettings.json file by default. However, you can also have the IConfiguration object retrieve values from other sources, including your Key Vault. Once you’ve integrated the Key Vault with your application’s IConfiguraiton object, you can selectively remove settings you want to keep secret from your appsettings.json file and move those settings to your Key Vault.

Currently, in my case study’s server-side frontend, I use this code to retrieve the App Registration that gives my frontend the necessary permissions to call my case study’s backend Web Service:

builder.Services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
    .AddMicrosoftIdentityWebApp(builder.Configuration)
    .EnableTokenAcquisitionToCallDownstreamApi()
    .AddInMemoryTokenCaches();

In that code, the AddMicrosoftIdentityWebApp method uses the IConfiguration object (retrieved from builder object’s Configuration property) to read the settings from my application’s appsettings.json file that I showed at the start of this post. If I integrate my Key Vault with my application’s IConfiguration object, the AddMicrosoftIdentityWebApp can also retrieve any settings it needs from the Key Vault.

Integrating the Key Vault into the IConfiguration Object

The first step in integrating the Key Vault and your IConfiguration object is to add the NuGet package Azure.Extensions.AspNetCore.Configuration.Secrets to your application. Once that’s in place, you can integrate your vault into your configuration settings with this code in your Program.cs file, put after the statement that calls your WebApplication.CreateBuilder method (you’ll need to replace the URL in this example with the URL for your vault from your vault’s Overview page):

Uri kvu = new Uri("https://warehousemgmtvault.vault.azure.net/");
builder.Configuration.AddAzureKeyVault(
    kvu,
    new DefaultAzureCredential()
);

Your next step is to cut your ClientSecret from your appsettings.json file’s AzureAd section, leaving the settings you don’t mind sharing:

"AzureAd": {
   "Instance": "https://login.microsoftonline.com/",
   "Domain": "warehousemgmtfrontendaspnet.azurewebsites.net",
   "TenantId": "e98…-…-…-…-…461",
   "ClientId": "80e0a00e-8cf2-4cfe-8a46-cc1dec17bf3b",
   "CallbackPath": "/signin-oidc"
 },  

Moving an App Setting to the Key Vault

The final step is to add the value of your ClientSecret setting to your Key Vault. To do that, surf to your vault in the Azure Portal and, in the menu on the left, drill down through Objects to select the Secrets node. At the left end of the menu across the top, click on the + Generate/Import choice to display the Create a secret page.

On that page you have to set the Name textbox to a modified version of the path to your ClientSecret setting. In your ASP.NET code, when using the IConfiguration object, the path to your ClientSecret setting in the AzureAd section of your appsettings.json file is AzureAd:ClientSecret. However, the Key Vault doesn’t permit colons in key names, so you’ll need to replace the colon with something else—two hyphens, for example. As a result, you would set the key’s name to AzureAd--ClientSecret.

Once you’ve set the name of your setting, in the Secret value textbox, paste your ClientSecret’s value before clicking the Create button to save your new key.

With those changes, your ClientSecret is no longer in your application’s source code and is only visible to Key Vault administrators. In your code, the IConfiguration object will fall back to a value in your Key Vault for any value it can’t find in appsettings.json.

Testing from the Desktop

If you created the Key Vault that your secrets are stored in (and/or have administrator permissions to that vault) and are accessing the Key Vault directly, then you should be able test your Key Vault–related code from your desktop. In real life (as opposed to this blog series, for example), one or more of those conditions is probably not the case and you won’t have access to the vault from your desktop.

You can test your app (with some limitations) from the desktop in Visual Studio, using its Secret Manager Tool. In Visual Studio Code, you can get equivalent functionality using Cloud Code for VS Code or the Doppler VS Code Extension (among other tools). Both Cloud Code and VS Code are available from Visual Studio Code’s Extension pane. I’ll just cover the Secret Manager that comes with Visual Studio.

To enable Secret Manager for a project in Visual Studio, right-click on your project in Solution Manager and select Manage User Secrets. That will open a tab with the skeleton of a JSON file named secrets.json. You can then add your secret to that file. For my Web Service with its DBIdentity secret, the entry would look something like this:

{  
   "DbIdentity":" b54…-…-…-…-…ef8"
}

You can then integrate your secrets.json file into your IConfiguration object by adding this line to your Program.cs file:

builder.Configuration.AddUserSecrets<Program>();

That will enable code that use the IConfiguration object to retrieve the value from your secrets.json file:

string? DbAccessIdentityId2 = app.Configuration["DbAccessidentity"];

There are several limitations with this approach, however, the first being that, if you’re using code to directly access the Key Vault or using the Environment object, that code will ignore the secrets.json file.

The second issue is that secrets.json doesn’t understand Key Vault references (i.e., if you use the Microsoft.KeyVault() function in your secrets.json file then app.Configuration returns the text @Microsoft.KeyVault(…and not the value from the Key Vault). That means that you’ll need to put the actual secret value in the secrets.json file (as in my example). And that means that you need to be careful about the secrets you put in the secrets.json file.

The good news here is that the secrets.json file is not stored in the folder with the rest of your project files and, as a result, won’t be pushed to your source control system so your settings won’t be shared with any other developer. The bad news is that the file is not encrypted and, as a result, is available to anyone who gains access to your files (or is looking over your shoulder). At the very least, you should only store non-production values in secrets.json.

Next Steps

An alternative to, for example, to using a “secret key” to claim an App Registration is to stop using a secret key and, instead, use a client certificate as your credentials. I’ll cover both storing and retrieving certificates in a server-side application in my next post. After that? Accessing your Key Vault from a client-side application.