Building Node.js Microservices: MySQL, MongoDB & RabbitMQ. In this blog post, we explore how to get started with microservices architecture in Node.js, including some example.
Microservices architecture is a method of organizing an application as a set of services based on business components or capabilities. The services are manageable, tested, and self contained. They are also loosely connected, allowing for autonomous deployment of modifications, and each service can be given to a small team of developers.
Let’s start the article Building Nodejs Microservices: MySQL, MongoDB & RabbitMQ.
Benefits of Adopting Microservices Architecture
Microservices have been effectively implemented by some of the world’s top corporations, including Amazon, Netflix, and Uber. These businesses have gradually disassembled their monolithic application into tiny services based on business skills such as search, buying, payment, orders, and so on, and allocated each service to a distinct team.
- Reducing code complexity: Microservices help to reduce code complexity by dividing the code into logical services.
- Shorter release cycles: It gives you the ability to scale selected services on demand with the help of cloud services like AWS, Azure, etc.
- Easy application management: It reduces the application size for each team, simplifying maintaining and releasing new features.
- Shorter build time: It reduces dependencies and enhances deployment flexibility to independently deploy only those services that are part of the release cycle.
If a company wishes to move from monolith to microservices, it should:
- Deploy new functionalities with zero downtime, especially for SaaS based organizations that require their product to be available at all times.
- Isolate certain data and processes to meet GDPR (General Data Protection Regulation) and SOC2 criteria.
- Possess a high level of team autonomy in which each team makes its own decisions and builds its software. The design of microservices follows a pattern that
The majority of startups or new projects within organizations begin with a monolithic application or a collection of tightly coupled tiny applications. The monolith may contain numerous features, but all of the programming logic for those features is contained in the application code. Because the code is so tightly knit, it’s difficult to unravel. Adding a new feature or modifying an existing one in a monolith can be time consuming and inhibit application functionality.
Characteristics of Microservices
There are four main characteristics of microservices as follows:
- Multiple Components In the microservices approach, the software can be built on small logical components so that each service can be easily modified and deployed without messing up the integrity of the entire application.
- Services Based on Business Capabilities The microservices approach is built upon business functions and precedence. In a monolith, all the business logic is built into a single application, and each team has a set of responsibilities like database, UIs, backend logic, etc. In microservices, each team is assigned to a separate business capability, which acts as a separate product like search, orders, payment, etc.
- Decentralized Microservices use various technologies and platforms. One service can use a different set of tools than others. Microservices also supports decentralized data management. Each service usually manages its database, which aligns with decentralized governance.
- Fault Resistant Since there are multiple sets of services, there is always a chance that one service can fail. However, unlike a monolith architecture, the probability of the failed service affecting other services is much lower. A good microservices architecture can withstand the failure of a service.
Building Node.js Microservices: Advanced Patterns and Event Driven Architecture
Following with Building Nodejs Microservices: MySQL, MongoDB & RabbitMQ, we use Node.js and TypeScript to develop two microservices for the sample project. One microservice is connected to a MySQL database, while the other is connected to a MongoDB database. Communication between the two microservices takes place via the AMQP (Advanced Message Queuing Protocol) protocol and RabbitMQ.
AMQP’s overall purpose is to facilitate message transit through broker services over TCP/IP connections. AMQP is called a compact protocol since it is a binary protocol, which means that all data sent over AMQP is binary data. A binary protocol prevents useless data from being sent across the wire.
The Admin application includes REST APIs for adding, updating, and deleting new objects. Admin also updates the overall number of items in an inventory. This information must also be updated individually in the cart service. The sample code for this example is provided at the end of this blog post.
Let’s install MySQL and MongoDB via various methods, such as XAMPP and MongoDB drivers, and then we install RabbitMQ drivers for Windows directly. However, in this example, we run them using Docker images, as detailed below:
Launch MySQL Database Container
First, we launch the MySQL docker image. If you do not already have the image installed, the following command will do so:
docker run --name=mysql-image -p3306:3306 -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql/mysql-server:8.0.20
Next, grant access to all IPs to the MySQL root user. To do so, first connect to the MySQL container using the following command:
docker exec -it mysql-image bash
After running the above command, log in to the MySQL shell as a root user:
mysql -u root -p
Once you are connected to the MySQL, create a database with the following command.
mysql> CREATE DATABASE db-admin;
Then, to make the MySQL docker image accessible to our application, you must specify the host to ‘%’:
mysql> update mysql.user set host = '%' where user='root';
Once you are done, you can proceed to the next step.
Launch MongoDB and RabbitMQ Container
Here, in this step run the following command to launch the MongoDB docker container.
docker run --name mongodb -d -p 27017:27017 mongo
You can now able to access the DB using the following URL mongodb://localhost:27017/main.
Start the RabbitMQ Docker container with the following command.
docker run -d --hostname my-rabbit --name some-rabbit -p 15672:15672 rabbitmq:3.8-management
The RabbitMQ dashboard should be accessible via the URL http://localhost:15672/. Guest is the username and password.
Create Admin Service
It’s a small NodeJS server that makes use of Typescript and TypeORM. The server is then linked to the MySQL database. This project will rely on the following dependencies.
"dependencies": {
"amqplib": "^0.7.1",
"cors": "^2.8.5",
"express": "^4.17.1",
"mysql": "^2.18.1",
"reflect-metadata": "^0.1.13",
"typeorm": "^0.2.31"
},
Let’s define the table and columns with an Item entity. The TypeORM DataMapper pattern will be used.
import {
Column,
CreateDateColumn,
Entity,
PrimaryGeneratedColumn,
} from "typeorm";
@Entity()
export class Item {
@PrimaryGeneratedColumn("uuid")
id: string;
@Column()
name: string;
@Column()
imageUrl: string;
@Column({ default: 0 })
totalItems: number;
@CreateDateColumn({ name: "createdAt", type: "datetime" })
createdAt: Date;
}
The admin service’s function is to save item details in a database. Because consumers will use the cart service to purchase things from the store, whenever the admin adds, edits, or deletes an item, the changes should also be reflected on the cart service.
We will also include the “ormconfig.json” file to provide MySQL database credentials.
{
"type": "mysql",
"host": "0.0.0.0",
"port": 3306,
"username": "rootall",
"password": "my-secret-pw",
"database": "db-admin",
"entities": [
"lib/entity/*.js"
],
"logging": false,
"synchronize": true
}
Following the addition of the database credentials, we connect to the database and AMQP in the “main.ts” file after installing the amqplib requirement.
createConnection()
.then((database: Connection) => {
const itemRepo = database.getRepository(Item);
amqp.connect("amqp://guest:guest@127.0.0.1:5672/", (error, connection) => {
connection.createChannel((channelConnectionError, amqpChannel) => {
if (channelConnectionError) {
throw new Error(channelConnectionError);
}
As demonstrated below, we establish an AMQP channel to push our items in Queue with a relevant queue name and data:
const ITEM_CREATED = "ITEM-CREATED";
const ITEM_UPDATED = "ITEM-UPDATED";
const ITEM_DELETED = "ITEM-DELETED";
Following that, we will implement REST API routes to allow the admin to do CRUD operations. We will additionally push data into a queue with relevant queue names and data in each create, update, and delete route. Cart service will absorb the queues at a later point.
app.post("/api/items", async (request: Request, response: Response) => {
const item = await itemRepo.create(request.body);
const result = await itemRepo.save(item);
amqpChannel.sendToQueue(
ITEM_CREATED,
Buffer.from(JSON.stringify(result))
);
return response.send(result);
});
app.put(
"/api/items/:id",
async (request: Request, response: Response) => {
const item = await itemRepo.findOne(request.params.id);
itemRepo.merge(item, request.body);
const result = await itemRepo.save(item);
amqpChannel.sendToQueue(
ITEM_UPDATED,
Buffer.from(JSON.stringify(result))
);
return response.send(result);
}
);
app.delete(
"/api/items/:id",
async (request: Request, response: Response) => {
const result = await itemRepo.delete(request.params.id);
amqpChannel.sendToQueue(
ITEM_DELETED,
Buffer.from(request.params.id)
);
return response.send(result);
}
);
amqpChannel.sendToQue will use the AMQP protocol to publish the data, and we will need to subscribe to the specified queue in our cart service to run independently.
Create Cart Service
The cart service’s role is to use the item and add it to the customer’s cart. In addition, the admin service will be in charge of adding, deleting, and updating goods in the cart service. The following dependencies are used in this service:
"dependencies": {
"amqplib": "^0.7.1",
"axios": "^0.21.1",
"cors": "^2.8.5",
"express": "^4.17.1",
"mongodb": "^3.6.5",
"reflect-metadata": "^0.1.13",
"typeorm": "^0.2.31"
},
The cart service will use the same project design as the admin service. Let’s also include an item entity in the cart service. The only difference between this object and others is the adminId field, which contains the id allocated to the admin item entry.
import { Column, CreateDateColumn, Entity, ObjectIdColumn } from "typeorm";
@Entity()
export class Item {
@ObjectIdColumn()
id: string;
@Column({ unique: true })
adminId: string;
@Column()
name: string;
@Column()
imageUrl: string;
@Column({ default: 0 })
totalItems: number;
@CreateDateColumn({ name: "createdAt", type: "datetime" })
createdAt: Date;
}
We will create the “ormconfig.json” file to add info about the MongoDB connection credentials.
{
"type": "mongodb",
"host": "0.0.0.0",
"database": "db-cart",
"synchronize": true,
"logging": true,
"entities": [
"lib/entity/*.js"
],
"cli": {
"entitiesDir": "lib/entity"
}
}
Next, you will connect to AMQP, which is hosted by rabbitMQ, in the “main.ts” file. Following a successful connection, we will subscribe to the Queues pushed by the admin service.
amqp.connect("amqp://guest:guest@127.0.0.1:5672/", (error, connection) => {
if (error) {
throw new Error(error.message);
}
connection.createChannel((channelConnectionError, amqpChannel) => {
if (channelConnectionError) {
throw new Error(channelConnectionError.message);
}
amqpChannel.assertQueue(ITEM_CREATED, { durable: false });
amqpChannel.assertQueue(ITEM_UPDATED, { durable: false });
amqpChannel.assertQueue(ITEM_DELETED, { durable: false });
Following that, we will listen to the specific queue to complete the task assigned to that queue. When data is pushed to the queue, we will add a consumer who will receive it. For example, in the case of the ITEM_CREATED queue, we will add a consumer as shown below, and when the queue is triggered, we will create a row in the database.
amqpChannel.consume(
ITEM_CREATED,
async (msg) => {
const createItemEvent: Item = JSON.parse(msg.content.toString());
const item = new Item();
item.adminId = createItemEvent.id;
item.name = createItemEvent.name;
item.imageUrl = createItemEvent.imageUrl;
item.totalItems = createItemEvent.totalItems;
await itemRepo.save(item);
console.log(ITEM_CREATED);
},
{ noAck: true }
);
Start Both Services
Let’s use the “yarn start” command to start both services. The admin service will be accessible via port 5001, while the cart service will be accessible via port 5002.
yarn start
We can now use the admin APIs. First, we will create an item using the POST /api/items API route.
If we update and delete items in the admin service, the cart service should be affected as well. Similarly, if we purchase the item from the cart, the item count in the admin database will be updated as well.
Both services communicate with one another using the AMPQ protocol and RabbitMQ.
Thank you for reading Building Nodejs Microservices: MySQL, MongoDB & RabbitMQ. We shall conclude the topic.
Building Node.js Microservices: MySQL, MongoDB & RabbitMQ Conclusion
The preceding example demonstrates the significance of microservices design. Although establishing a microservices project increases complexity at first, it benefits scalability, testability, and maintaining low coherence in the application code. It can also give the optimal architecture for continuous delivery with on demand scaling for certain services.
Microservices design is directly aligned with business requirements and can be broken down into small services based on business capabilities. We also built a simple Event Driven microservice architecture to show how different services might connect with one another. The majority of the time, event driven architecture is employed to communicate between disconnected services. The Event Driven method has several advantages, including selectable scalability and independent service failure due to the architecture’s decoupling.
