5 Mistakes Teams Make When Scaling  .NET Applications

Avoid these common infrastructure-related mistakes when you’re ready to scale your .NET app to keep your performance at scale too.

IT teams often assume that scaling a .NET application means moving it to a larger virtual machine or containerizing the application. While such actions do have their place, performance problems are often tied to infrastructure design rather than the application itself.

This article discusses some of the most common infrastructure-related mistakes that are often made when scaling a .NET application.

1. Adding Instances Without Thinking About Traffic Distribution

Application scaling very often involves adding additional nodes to handle the increased demand. However, it is a mistake to add nodes without considering how requests will reach those nodes. Oftentimes organizations will use round-robin load balancing to evenly distribute inbound requests across the available nodes. However, load balancing can break down when session affinity comes into play.

Session affinity occurs as a result of session data being stored on a specific server. This in turn causes all of a user’s requests to be sent to that same server, because that’s where the user’s session lives. This isn’t necessarily a problem by itself, but session affinity can become problematic under the right conditions, especially when there is a mixture of active and inactive users.

In such situations, a particular instance may become overloaded, even if plenty of capacity exists on other instances, simply because the session is bound to that instance. Additionally, if that instance were to fail, then the users on that instance lose their session.

A better approach is to design services to be stateless and to store session data within shared storage rather than placing it in an instance’s memory. This allows the load balancer to freely distribute requests.

2. Assuming That All Requests Look Alike

Basic load distribution strategies, such as round robin, assume that every instance can deliver the same level of performance and that every request has the same impact. In reality though, these assumptions are naïve.

Some requests, for example, trigger longer running operations, call external services or make more complex database queries. As such, some requests will inevitably be completed more quickly than others. This can result in some nodes being bogged down while others remain nearly idle.

The best way to avoid this type of problem is to avoid blindly distributing traffic across nodes. Instead, consider each node’s health when making routing decisions. As an example, if a node were experiencing excessive latency or had a significant queue depth, then it would be better to route inbound requests to a node that is under less of a load rather than assigning the request to the struggling node just because it is that node’s turn.

3. Not Having a Load-Shedding Strategy

Although it may seem counterintuitive, load shedding can sometimes be the most appropriate course of action. For those who might not be familiar with the term, load shedding refers to the practice of intentionally rejecting certain requests as a way of helping the system to stay healthy.

To use an analogy from everyday life, consider what might happen if a restaurant kept seating additional guests in spite of the kitchen and the waitstaff being completely overwhelmed. In such a situation, everyone would receive poor service. Conversely, if the restaurant were to pause and not seat anyone else until the staff had a chance to catch up, then it would result in a better overall experience for everyone.

Load shedding works similarly. When a system becomes overwhelmed with more requests than the nodes can handle, then the load balancer intentionally fails some requests so that those requests that do make it through can still be serviced normally.

4. Lack of Visibility

This one seems so simple (and it is), yet it is so important. Organizations often make the mistake of just assuming that load balancing is working as intended without doing anything to verify their assumptions. The problem is that if an organization cannot see how requests are being routed, then scaling an application becomes guesswork.

Without proper visibility, you can’t tell whether traffic is being evenly distributed, if there are nodes that are constantly struggling or which instances are handling slow requests. It becomes easy to misdiagnose bottlenecks and blame the application for poor performance when routing is the real problem (or vice versa).

Ideally, an organization should be able to collect per-instance metrics from its nodes including CPU and memory usage and the request count. There should also be a way to trace requests as they flow across services and there should be a way to track load distribution metrics to see how the individual nodes are being leveraged.

5. Treating Load Balancing as an Afterthought

One of the single biggest mistakes that is often made when scaling an application is treating load balancing as an afterthought. Typically, an application is deployed and works fine. Eventually though, demand increases and the application begins to struggle. IT distributes the application across several nodes and places a basic load balancer in front of those nodes. In these types of situations, load balancers often use default settings because, “round robin is good enough for not” or “the cloud provider’s defaults should be fine.”

The end result is that while the application has indeed been scaled, there is a good chance that some of the original problems still exist. The added infrastructure now makes those problems more difficult to diagnose.

True load balancing is not about spreading traffic evenly. It’s about making informed decisions as to where each inbound request should be routed. This informed decision making means using application aware routing to base routing decisions on application behavior, not just routing rules. It also means taking instance health and readiness into account and continually adjusting routing decisions based on latency, errors and available capacity.