Micro services have become widely adopted by many companies and as a result of this, a lot of developers have gone into learning how to develop this type of applications. When I first started building micro services and cloud native applications I was tasked to add a caching system to a project I was working on. I ignorantly choose to use embedded cache. During test for scalability, I realized my system wasn't built to scale horizontally.

While researching how I could solve the problem, I came across two terms (stateful & stateless micro services) which completely changed how I think about scalability when designing micro services and specifically what makes a micro service horizontally scalable. In this blog post, we're going to talk about these two terms and a few things to keep in mind when designing and building a micro service application.

What is a Stateful & Stateless Microservice?

These two terms are not often talked about when defining microservices as they could bring a lot of confusion. However a foundational understanding of them helps in the development of quality and scalable microservice applications.

Stateful Microservice:

A stateful microservice is a service which keeps and tracks the previous state of a request on it's server rather than on an external server/service. For example using an embedded caching system like I did, when a user sends a request to retrieve an information from the server, the server retrieves this information from a DB and stores it on the server and when the user requests for the same information again, the server retrieves it from it's previous stored state.

This is advantageous as it reduces the overhead cost of reading the information from an external server however on a microservice context this completely makes your application incapable of scaling horizontally.

Imagine a situation where you have to spin up a new server in your infrastructure to better distribute loads/traffic evenly across all your servers. When these servers are stateful, you face the risk of data inconsistency because when users gets routed to the new instantiated servers they start on a fresh state thus losing other data or sessions stored in the previous servers. Most engineers would replicate the server but for me this isn't a great idea.

Stateless Microservice:

From the definition of a stateful service, one can easily guess that a stateless microservice is a service which keeps no states of it's previous requests or stores data in memory or in the server. It instead uses an external service like a DB (which is stateful) to store states or records required by the service. Microservices are born stateless which makes them very easy to scale horizontally while maintaining data consistency across all servers. Requests to the server are self-contained and it requires no session storage or embedded caching system for the server to efficiently process the request and return a response.

So how then did I implement a caching system to my service? I used a distributed caching system which sits across multiple different servers. This enabled me to achieve better fault tolerance, availability and scalability of my services without worrying about data consistency.

What it means to build a horizontally scalable application

Building a horizontally scalable microservice application means designing a stateless application capable of scaling and maintaining data consistency across multiple server instances of the same service. This is not achievable with monolithic applications hence one of the numerous reasons why micro services are widely adopted in the industry today.

Things to keep in mind when designing a micro service application

• Decide early on if you want your application to be stateful or stateless.

• If possible avoid designing/building a stateful micro service application unless you must.

• Without session usage, use Authentication and Authorization to connect to the servers.

• Ensure to use REST design patterns for building out your application

• Avoid using sessions as it's not possible to scale sessions efficiently. If using sessions, replicate all sessions to all servers at the same time.

Conclusion

When designing for scalability in micro services, it's important to keep in mind the principles and best practices which makes up a highly scalable micro service application. It can be tedious but with a fundamental understanding of it any developer can build a system that is fault tolerant and is highly available.