In this article by David Gonzalez, author of the book Developing Microservices with Node.js, we will cover the need for microservices, explain the monolithic approach, and study how to build and deploy microservices.
(For more resources related to this topic, see here.)
Need for microservices
The world of software development has evolved quickly over the past 40 years. One of the key points of this evolution has been the size of these systems. From the days of MS-DOS, we taken a hundred-fold leap into our present systems. This growth in size creates a need for better ways of organizing the code and software components. Usually, when a company grows due to business needs, which is known as organic growth, the software gets organized on a monolithic architecture as it is the easiest and quickest way of building software. After few years (or even months), adding new features becomes harder due to the coupled nature of the created software.
Monolithic software
There are a few companies that have already started building their software using microservices, which is the ideal scenario. The problem is that not all the companies can plan their software upfront. Instead of planning, these companies build the software based on the organic growth experienced: few software components that group business flows by affinity. It is not rare to see companies having two big software components: the user facing website and the internal administration tools. This is usually known as a monolithic software architecture.
Some of these companies face big problems when trying to scale the engineering teams. It is hard to coordinate the teams that build, deploy, and maintain a single software component. Clashes on releases and reintroduction of bugs are a common problem that drains a big chunk of energy from the teams. One of the solution to this problem (it also has other benefits) is to split the monolithic software into microservices so that the teams are able to specialize in few smaller modules and autonomous and isolated software components that can be versioned, updated, and deployed without interfering with the rest of the systems of the company.
One of the most interesting solutions to this problem is splitting the monolithic architecture into microservices. This enables the engineering team to create isolated and autonomous units of work that are highly specialized in a given task (such as sending e-mails, processing card payment, and so on).
Microservices in the real world
Microservices are small software components that specialize in one task and work together to achieve a higher-level task. Forget about software for a second and think about how a company works. When someone applies for a job in a company, he applies for a given position: software engineer, systems administrator, or office manager The reason for it can be summarized in one word—specialization. If you are used to working as a software engineer, you will get better with the experience and add more value to the company. The fact that you don’t know how to deal with a customer, won’t affect your performance as it is not your area of expertise and will hardly add any value to your day-to-day work.
A microservice is an autonomous unit of work that can execute one task without interfering with other parts of the system, similar to what a job position is to a company. This has a number of benefits that can be used in favor of the engineering team in order to help to scale the systems of a company.
Nowadays, hundreds of systems are built using a microservices-oriented architectures, as follows:
- Netflix: They are one of the most popular streaming services and have built an entire ecosystem of applications that collaborate in order to provide a reliable and scalable streaming system used across the globe.
- Spotify: They are one of the leading music streaming services in the world and have built this application using microservices. Every single widget of the application (which is a website exposed as a desktop app using Chromium Embedded Framework (CEF)) is a different microservice that can be updated individually.
First, there was the monolith
A huge percentage (my estimate is around 90%) of the modern enterprise software is built following a monolithic approach.
Huge software components that run in a single container and have a well-defined development life cycle that goes completely against the following agile principles, deliver early and deliver often (https://en.wikipedia.org/wiki/Release_early,_release_often):
- Deliver early: The sooner you fail, the easier it is to recover. If you are working for two years in a software component and then, it is released, there is a huge risk of deviation from the original requirements, which are usually wrong and changing every few days.
- Deliver often: Everything of the software is delivered to all the stake holders so that they can have their inputs and see the changes reflected in the software. Errors can be fixed in a few days and improvements are identified easily.
Companies build big software components instead of smaller ones that work together as it is the natural thing to do, as follows:
- The developer has a new requirement.
- He builds a new method on an existing class on the service layer.
- The method is exposed on the API via HTTP, SOAP, or any other protocol.
Now, repeat it by the number of developers in your company and you will obtain something called organic growth. Organic growth is the type of uncontrolled and unplanned growth on software systems under business pressure without an adequate long-term planning, and it is bad.
How to tackle the organic growth?
The first thing needed to tackle the organic growth is make sure that business and IT are aligned in the company. Usually, in big companies, IT is not seen as a core part of the business.
Organizations outsource their IT systems, keeping the cost in mind, but not the quality so that the partners building these software components are focused on one thing: deliver on time and according to the specification, even if it is incorrect.
This produces a less-than-ideal ecosystem to respond to the business needs with a working solution for an existing problem. IT is lead by people who barely understand how the systems are built and usually overlook the complexity of the software development.
Fortunately, this is a changing tendency as IT systems have become the drivers of 99% of the businesses around the world, but we need to be smarter about how we build them.
The first measure to tackle the organic growth is to align IT and business stakeholders in order to work together, educating the non-technical stakeholders is the key to success.
If we go back to the example from the previous section (few releases with quite big changes). Can we do it better?
Of course, we can. Divide the work into manageable software artifacts that model a single and well-defined business activity and give it an entity.
It does not need to be a microservice at this stage, but keeping the logic inside a separated, well-defined, easy testable, and decoupled module will give us a huge advantage towards future changes in the application.
Building microservices – The fallback strategy
When you design a system, we usually think about the replaceability of the existing components. For example, when using a persistence technology in Java, we tend to lean towards the standards (Java Persistence API (JPA)) so that we can replace the underneath implementation without too much effort.
Microservices take the same approach, but they isolate the problem instead of working towards an easy replaceability.
Also, e-mailing is something that, although it seems simple, always ends up giving problems.
Consider that we want to replace Mandrill with a plain SMTP server, such as Gmail. We don’t need to do anything special, we just change the implementation and rollout the new version of our microservice, as follows:
var nodemailer = require('nodemailer');
var seneca = require("seneca")();
var transporter = nodemailer.createTransport({
service: 'Gmail',
auth: {
user: '[email protected]',
pass: 'verysecurepassword'
}
});
/**
* Sends an email including the content.
*/
seneca.add({area: "email", action: "send"}, function(args, done) {
var mailOptions = {
from: 'Micromerce Info ✔ <[email protected]>',
to: args.to,
subject: args.subject,
html: args.body
};
transporter.sendMail(mailOptions, function(error, info){
if(error){
done({code: e}, null);
}
done(null, {status: "sent"});
});
});For the outer world, our simplest version of the e-mail sender is now at all lights, using SMTP through Gmail to deliver our e-mails.
We could even rollout one server with this version and send some traffic to it in order to validate our implementation without affecting all the customers (in other words, contain the failure).
Deploying microservices
Deployment is usually the ugly friend of the software development life cycle party. There is a missing contact point in between development and system administration, which DevOps is going to solve in the following few years (or has already done it and no one told me).
The following is the graph showing the cost of fixing software bugs versus the various phases of development:
From the continuous integration up to continuous delivery, the process should be automated as much as possible, where as much as possible means 100%. Remember, humans are imperfect…if we rely on humans carrying on a manual repetitive process for a bug-free software, we are walking the wrong path.
Remember that a machine will always be error free (as long as the algorithm that is executed is error free) so…why not let a machine control our infrastructure?
Summary
In this article, we saw how microservices are required in complex software systems, how the monolithic approach is useful, and how to build and deploy microservices.
Resources for Article:
Further resources on this subject:
- Making a Web Server in Node.js [article]
- Node.js Fundamentals and Asynchronous JavaScript [article]
- An Introduction to Node.js Design Patterns [article]
