Turning Methods Into Benchmarks

Benchmarking is an increasingly common topic in development because it allows companies to measure the performance of their applications, especially when new functions are added. Learn in this article how to implement benchmarking in a .NET application.

When implementing new functions in a system, it is crucial to check the performance of the application as a whole to see whether changes made to the code have improved or degraded performance. To help the developer in this analysis, there are great tools that are easy to implement and have many features.

In this article, we will see a brief introduction to the benchmarking concept and we will implement in practice an application applying the performance analysis through the BenchmarkDotNet library.

Benchmarking Concept

The term “benchmark” in computing is the action of executing a computer program, a set of programs or other operations with the aim of evaluating the relative performance of an object, usually by running a series of standard tests on it.

Normally, benchmarking is associated with evaluating the performance of computer hardware, but in many cases this technique is also applicable to software. Software benchmarks can be found in compilers, logic execution or database management systems.

Why Is Benchmarking Useful?

Implementing a benchmark in a system allows us to have a detailed performance analysis and ensure reliable and accurate results.

In this context, when developing an application, we may need to conduct performance checks in the execution of the software. To help with this task, the use of benchmarking becomes essential, so we have access to the performance metrics of the methods, in addition to being able to verify if the changes made to the code improved or worsened performance.

BenchmarkDotNet Library

One of the advantages of programming in .NET is the wide variety of NuGet libraries available for any need. In the context of benchmarking, there is a great tool to help developers work—the BenchmarkDotNet library.

BenchmarkDotNet is a lightweight, open-source and very powerful library that can turn your methods into benchmarks, track the performance and share the captured data, where everything is done in a simple and friendly way.

Implementing BenchmarkDotNet in a .NET Application

To demonstrate the use of BenchmarkDotNet, we will create a simple console application, implement some methods, run the application and verify the analysis performed by BenchmarkDotNet.

You can access the complete source code of the final project at this link: source code.

So to create the base app, run the following command in the console:

dotnet new console -n BenchmarkConceptApp

Then you can open the project with your favorite IDE. In this article, we will use Visual Studio 2022.

Adding BenchmarkDotNet Dependencies

First let’s add the dependency of the “BenchmarkDotNet” library. For that, double-click on the project (BenchmarkConceptApp.csproj) and add the code below:

<ItemGroup>
<PackageReference Include="BenchmarkDotNet" Version="0.13.1" />
</ItemGroup>

Then rebuild the solution. This way we already have the necessary dependencies to implement the benchmark.

Creating the Class and the Methods

Inside the project, create a new class called “CryptoComparison” and then replace the generated code with the code below:

using BenchmarkDotNet.Attributes;
using System.Security.Cryptography;

namespace BenchmarkConceptApp
{
    [RankColumn]
    [MemoryDiagnoser]
    public class CryptoComparison
    {
        private const int itemQuantity = 10000;
        private readonly byte[] data;

        private readonly SHA256 sha256 = SHA256.Create();
        private readonly MD5 md5 = MD5.Create();

        public CryptoComparison()
        {
            data = new byte[itemQuantity];
            new Random(42).NextBytes(data);
        }

        [Benchmark]
        public byte[] Sha256() => sha256.ComputeHash(data);

        [Benchmark]
        public byte[] Md5() => md5.ComputeHash(data);
    }
}

In this class, we add two attributes: RankColumn and MemoryDiagnoser.

• RankColumn – Creates a column called “Rank,” which will indicate which were the fastest and slowest methods when the results are generated.
• MemoryDiagnoser – Performs memory allocation diagnostics during code execution.

We also added variables with random values and two methods that calculate the hash value for the specified byte array, in SHA256 and MD5 formats.

Configuring the Program Class

The last step is to configure the initialization of the app so that the class that contains the previously created methods is executed and analyzed by the Benchmark. This is very simple, just add the code below in the Program class.

using BenchmarkConceptApp;
using BenchmarkDotNet.Running;

BenchmarkRunner.Run<CryptoComparison>();

The only configuration we did was to add the Benchmark class responsible for executing “BenchmarkRunner,” calling the “Run” method and passing as a parameter the class that contains the methods to be analyzed.

Running the BenchmarkRunner

Now everything is ready for us to run the code and check the results obtained. To do this, just run the commands below in the terminal, inside the root folder of the project.

dotnet run -project BenchmarkConceptApp.csproj -c Release

Note that with this command we are running the application in “Release” mode because BenchmarkDotNet requires it to be that way so that it can do the analysis. If it is run in debug mode, an error will be generated. This is because in release mode the C# compiler makes some optimizations that are not available in debug mode.

Then, the analysis processing will be performed, which may take a few minutes. At the end of the execution, we will have the following results displayed in the terminal:

As can be seen in the results table the fastest method was Md5, which in the arithmetic mean (Mean column) took only 17.54 us (microseconds) compared to 40.29 us for the Sha256 method. It is also possible to confirm which one was the fastest through the “Rank” column, which shows that Md5 was in the first place.

BenchmarkDotNet will also generate log files and HTML with the results inside a folder called “BenchmarkDotNet.Artifacts” in the project root.

Conclusion

In this article, we saw the concept of benchmarking and how it fits perfectly into software development in the .NET universe. We also created an application to test the BenchmarkDotNet library in practice. We have seen that BenchmarkDotNet has many features to facilitate data performance development and analysis.

For complete coverage of all the functionality of this library, I suggest you read the documentation on the official website. There are many resources that can help you solve everyday problems.