Real-Time Communication Techniques

We explore the need for real time communications and hurdles, as well as the techniques/frameworks that help.

Life happens in real time and information exchange should be the same way. Turns out, this is easier said than done. Real-time communication demands have fundamentally changed how we build web and connected client applications. However, it is the promise of technology to turn problems into opportunities.

Let's start with some history and explore how we've enabled real-time communications between computer systems. As information protocols and servers/clients have gotten smarter, developers have been able to leverage sophisticated techniques to build for real-time information exchange. The end result is better connected apps and near real-time decision making—no one doubts the benefits.

Real-Time Needs

Us: Let's slow down. What's the rush?
Life: Hold my beer. Information is power.

Turns out, there are countless modern applications that don't just benefit but actively depend on real-time communications to be functional. Here are a few application types:

1. Chat... Duh!
2. Stock Tickers
3. Live Auctions/Polls
4. Sports/News Updates
5. Online Collaboration
6. Multiplayer Games
7. Location Services
8. Progress Indicators

And many, many more. Age-old request-response communication techniques often do not scale to serve the real-time needs of modern connected clients. When the need is real, technology must deliver real-time communications. The good news is that despite the hurdles, the solutions are plentiful and quite brilliant.

The Hurdle

1989 @ CERN—The European Particle Physics Lab. Sir Tim Berners-Lee started the basics of what would become the modern web today: a server and a client. More importantly, as a part of the World Wide Web Consortium (W3C), the information exchange standard was established. Behold the birth of the mighty Hypertext Transfer Protocol, or as it's better known today, HTTP.

The web as we use it today is rather different from the '90s. However, HTTP, which is an application protocol for distributed collaborative information systems, remains the foundation of data communication for the World Wide Web. HTTP has evolved over time, with the HTTP/2 standard focusing on security, data compression, better efficiency and lower latency. However, the core of HTTP communication has not changed a whole lot, it is still a request/response protocol. A client has to ask for information and the server then responds, which presents a fundamental challenge to real-time communications.

The Solutions

Despite the obvious hurdle, the need for real-time communication is nothing new. And for years, we've tried circumventing HTTP roadblocks in interesting ways. Let's do a quick rundown of the most popular techniques, each with a real-world anecdote for easier explanation.

Regular Polling

Anecdote: Ever done a long drive with little kids? Just a few minutes on the road and the kids will already be asking, "Are we there yet?" You respond kindly, but the question continues to be asked all through repeatedly. Parents worldwide will empathize.

In regular polling, the client repeatedly asks the server if there is any new information to be shared. The interval of requests can vary based on app needs and the server mostly responds with no new information. This is clearly not real time, but could have some benefits if the polling interval is short. Bandwidth misuse isn't a huge concern since most request-responses do not carry much data, but regular polling does involve repeated unnecessary round trips between client-server.

Long Polling

Anecdote: Picture another long drive with your toddler. This time though, when asked, 
"Are we there yet?', you simply stay quiet and do not respond until you get to your destination. Or you are forced to respond when you feel a tantrum is oncoming.

Long polling is an advanced form of polling, catering to real-time communications. The client makes a request for information up to the server. The server simply sits on the request until something noteworthy happens, or the request is about to time out. If the server does respond to prevent a timeout, the client can be programmed to fire back another request right away. This guarantees that the server has an open request to fire off a response at all times and can pass along information in real time. While long polling continues to be popular, it often necessitates custom orchestration at both the server and client level to be successfully implemented.

Server-Sent Events

Anecdote: You are shopping online on a popular retail website and agree to an innocuous terms and conditions checkbox. Then bam! You are flooded with fragments of marketing emails thrice a day, with no easy way to unsubscribe.

Server-Sent Events facilitate real-time communication, but is largely one directional push from server to client. An EventSource object (standardized in HTML5) is used client-side to capture streaming notifications from the server.

Forever Frames

Anecdote: You have had a not-so-fun breakup, yet part of you doesn't want to forget your ex-boyfriend/girlfriend, perhaps out of curiosity. You set out a secret personal time each day to check on your ex's social presence updates.

Forever Frame is a mostly IE-specific technique of helping in real-time communications, and involves a hidden iFrame on client-side that maintains a connection to server through scripting. The server sends down chunks of information that is received within the iFrame and managed by custom code on client side.

WebSockets

Anecdote: You have learned how to say 'Hello' in a foreign language before visiting the country where that language is spoken. On your first interaction, someone greets you and you gleefully respond back in the native language. The floodgates have opened and you are hit with a barrage of local language, whether you understand it or not.

With HTML5 standardization, WebSockets have emerged as a wonderful technique (read: magical unicorn stuff) to facilitate real-time communications. At the start, the server and client do a little handshake to make sure they can both speak the same language over WebSockets. Once negotiations are in place, WebSockets enable true persistent connection for bi-directional communications—the server can call the client and the client can call the server at any time. WebSockets now enjoy wide platform support, thus enabling real-time information exchange with low latency.

Putting It All Together

So, clearly there are several techniques to get around HTTP hurdles and facilitate real-time communications. The problem is that most of these techniques take a bit of work from the developer. Network stack is tricky business. Wouldn't it be nice if there were frameworks that abstracted away the communications complexity, so developers could focus on building real-time apps? Thankfully, they do exist—and they're open-source and quite brilliant.

SignalR

SignalR facilitates adding real-time communication to web applications running ASP.NET and connected clients across wide variety of platforms. While SignalR started years back with ASP.NET MVC, the latest reboot is called SignalR Core—it runs on ASP.NET Core and brings a ton of maturity.

SignalR provides APIs for bidirectional Remote Procedure Calls (RPCs) between server and client and abstracts away real-time communication complexities. While the SignalR hosting app is to be written in ASP.NET Core, the connected clients can be supported in various programming languages, like JavaScript, .NET, Java, C++ and mobile platforms. The preferred form of real-time transport, as well as backup techniques, are chosen specifically for given client-server pair.

Developers benefit from SignalR providing a uniform API canvas for connection and client management, as well as scaling to handle increased traffic. SignalR uses the concept of hubs on the server side to facilitate real-time communication and management of connected clients. The server and client can seamlessly invoke methods on each other, and such interactions can be strongly typed. While text-based JSON format is the default, SignalR also supports Messagepack protocol—binary serialization/deserialization of data for added efficacy.

SignalR is all about real-time communication, surfacing data changes as quickly as possible to everyone connected. And this works even if you are using polished complex UI, like DataGrids in Kendo UI and Telerik UI for ASP.NET MVC or ASP.NET Core. The underlying DataSource object used for object binding facilitates the communication with SignalR hubs. Don't worry, we've got you covered.

gRPC

gRPC is a high performance universal RPC framework, initially developed at Google. The goal here as well, is to abstract complexities at network layer and allow developers to focus on building real time apps.

gRPC automatically generates idiomatic cross-platform client & server bindings for variety of languages and platforms. gRPC service definition and format of information exchange is Protocol Buffers - a powerful binary serialization/deserialization toolset & language. gRPC, out of the box, provides rich features such as integrated authentication, bi-directional streaming and flow control.

Wrap Up

Life is short and information has to be shared real time. While the fundamentals of HTTP provided initial roadblocks, intelligent technical solutions soon found ways around. Today, we have a plethora of powerful and reliable real-time communication techniques, like WebSockets, Server-Sent Events and Long Polling. And open-source frameworks like SignalR and gRPC abstract away much of the network layer complexity. Developers are left with real-time app opportunities where the sky is the limit. Onward  and upwards to the next amazing 'Hello World' real-time chat application!