From Monoliths To Microservices

How do you detangle a monolithic system and migrate it to a microservices architecture? How do you do it while maintaining business-as-usual? As a companion to Building Microservices, this new book details multiple approaches for helping you transition from existing monolthic systems to microservice architectures. This book is ideal if you're looking to evolve your current systems, rather than just rewriting everything from scratch.

From monoliths to microservices

A monolithic application is built as a single unified unit while a microservices architecture is a collection of smaller, independently deployable services. Which one is right for you? It depends on a number of factors.

Netflix became one of the first high-profile companies to successfully migrate from a monolith to a cloud-based microservices architecture. It won the 2015 JAX Special Jury award in part due to this new infrastructure that internalized DevOps. Today, Netflix has more than a thousand microservices that manage and support separate parts of the platform, while its engineers deploy code frequently, sometimes thousands of times each day.

Organizations can benefit from either a monolithic or microservices architecture, depending on a number of different factors. When developing using a monolithic architecture, the primary advantage is fast development speed due to the simplicity of having an application based on one code base.

Atlassian followed the path to microservices in 2018 after we faced growing and scaling challenges with Jira and Confluence. We found that our single-tenant, monolithic architectures running on premise would not be able to scale to future needs.

Microservices are by no means a silver bullet, but they solve a number of problems for growing software and companies. Since a microservices architecture consists of units that run independently, each service can be developed, updated, deployed, and scaled without affecting the other services. Software updates can be performed more frequently, with improved reliability, uptime, and performance. We went from pushing updates once a week, to two to three times a day.

As Atlassian grows, microservices enable us to scale teams and geographic locations more reliably by splitting along lines of service ownership. Before we started Vertigo, Atlassian had five different development centers around the world. These distributed teams were constrained by a centralized monolith and we needed to support them in an autonomous fashion. Microservices allow us to do so.

Plus, more generally, microservices make it easier for teams to update code and accelerate release cycles with continuous integration and continuous delivery (CI/CD). Teams can experiment with code and roll back if something goes wrong.

When we moved from a small number of monolithic codebases to many more distributed systems and services powering our products, unintended complexity arose. We initially struggled to add new capabilities with the same velocity and confidence as we had done in the past. Microservices can add increased complexity that leads to development sprawl, or rapid and unmanaged growth. It can be challenging to determine how different components relate to each other, who owns a particular software component, or how to avoid interfering with dependent components.

Many projects initially start out as a monolith and then evolve into a microservice architecture. As new features are added to a monolith, it may start to become cumbersome to have many developers working on a singular codebase. Code conflicts become more frequent and the risk of updates to one feature introducing bugs in an unrelated feature increases. When these undesirable patterns arise, it may be time to consider a migration to microservices.

In January 2016, we had about 15 total microservices. Now we have more than 1300. We moved 100K customers to the cloud, built a new platform along the way, transformed our culture, and ended up with new tools. We have happier, autonomous teams and a better DevOps culture.

Chandler Harris is a marketing strategist and writer for Atlassian. He has written for more than 40 different publications on subjects ranging from technology, science, business, finance, and education.

Migrating a monolithic system to an ecosystem of microservices is an epic journey. The ones who embark on this journey have aspirations such as increasing the scale of operation, accelerating the pace of change and escaping the high cost of change. They want to grow their number of teams while enabling them to deliver value in parallel and independently of each other. They want to rapidly experiment with their business's core capabilities and deliver value faster. They also want to escape the high cost associated with making changes to their existing monolithic systems.

Deciding what capability to decouple when and how to migrate incrementally are some of the architectural challenges of decomposing a monolith to an ecosystem of microservices. In this write-up, I share a few techniques that can guide the delivery teams - developers, architects, technical managers - to make these decomposition decisions along the journey.

As a founding principle the delivery teams need to minimize the dependencies of newly formed microservices to the monolith. A major benefit of microservices is to have a fast and independent release cycle. Having dependencies to the monolith - data, logic, APIs - couples the service to the monolith's release cycle, prohibiting this benefit. Often the main motivation for moving away from the monolith is the high cost and slow pace of change of the capabilities locked in it, so we want to progressively move in a direction that decouples these core capabilities by removing dependencies to the monolith. If the teams follow this guideline as they build out capabilities into their own services, what they find is instead, dependencies in the reverse direction, from the monolith to the services. This is a desired dependency direction as it does not slow down the pace of change for new services.

I am assuming that at this point the delivery teams are comfortable with building microservices and ready to attack the sticky problems. However they may find themselves limited with the capabilities that they can decouple next without a dependency back to the monolith. The root cause of this, is often a capability within the monolith that is leaky, not well defined as a domain concept, with many of the monolith capabilities depending on it. In order to be able to progress, the developers need to identify the sticky capability, deconstruct it into well defined domain concepts and then reify those domain concepts into separate services.

Developers can incrementally extract microservices from the sticky capability, one service at time. As an example, refactor 'customer wish list' first and extract that into a new service, then refactor 'customer payment preferences' into another microservice and repeat.

Most decoupling attempts start with extracting the user facing components and a few facade services to provide developer friendly APIs for the modern UIs, while the data remains locked in one schema and storage system. Though this approach gives some quick wins such as changing the UI more frequently, when it comes to core capabilities the delivery teams can only move as fast as the slowest part, the monolith and its monolithic data store. Simply put, without decoupling the data, the architecture is not microservices. Keeping all the data in the same data store is counter to the Decentralized Data Management characteristic of microservices.

As you break apart the monolith, a lot of code will be moved away from it and into new microservices. A monorepo helps you keep track of these kinds of changes. In addition, having everything in one place can help you recover from failures more quickly.

OK, this one only applies if you plan to use containers or Kubernetes for the microservices. In that case, containerization can help you homogenize your infrastructure. The monolith-in-a-box pattern consists of running the monolith inside a container such as Docker.

Because feature flags allow us to deploy inactive code to production and toggle it at any time, we can decouple feature releases from actual deployment. This gives developers an enormous degree of flexibility and control.

A modular monolith is the next best thing to microservices and a stepping stone towards them. The rule is that modules can only communicate over public APIs and everything is private by default. As a result, the code is less intertwined, relationships are easy to identify, and dependencies are clear-cut.

You will find that, in some cases, changes in one module propagate into others as you refactor the monolith. To combat this, you can create a translation layer between rapidly-changing modules. This anticorruption layer prevents changes in one module from impacting the rest.

The superpower microservices give you is the ability to deploy any microservice at any time with little or no coordination with other microservices. This is why data coupling must be avoided at all costs, as it creates dependencies between services. Each microservice must have a private and independent database.

I had tears forming in my eyes while reading this extremely familiar and lovely article. This is not academic, it could only have been written by someone who practically and pragmatically moved a real-world monolith to microservices under operational conditions.

Nice articles, but may i know how to handles typical SQL join across different databases? Like let say user databases and order databases and its payments databases. In monolith it will just doing three join, but how to achieve this in microservices with split database?

Each microservice should be able to operate independently else there would be coupling. The way this would be achieved is that each microservice should contain all the data it needs in its own database. Data that are relevant for one microservice that can only be gathered by another microservice should be emitted as events and consumed by the microservices that need them and stored locally on their databases. 041b061a72