Redis Cluster: How to Build a Scalable Redis Cache Infrastructure. In this post, we introduce cluster, its working principle, advantages, and limitations then show you how to create it and scale it on Linux.
Redis Cluster is a distributed version of the popular in memory database Redis. It provides the capability to scale Redis installations to handle larger volumes of data and higher traffic loads. While still under development, the Cluster of Redis has matured into a reliable technology suitable for production environments.
Shall we start with article Redis Cluster: How to Build a Scalable Redis Cache Infrastructure?
What is Redis Cluster?
Redis Cluster is a collection of interconnected Redis nodes designed to enable data sharing among them. Cluster nodes are responsible for specific data subsets within the cluster. It then distributes data across the appropriate nodes automatically when you write it. Similarly, when data is read from a Redis Cluster, it is retrieved from the node that holds the relevant data.
The primary distinction between Cluster in Redis and Redis is the distributed nature of Redis Cluster. In Cluster scenario, the distribution of data across multiple nodes is preferred over storing it on a single node. This inherent distribution enhances the scalability and it’s reliability compared to Redis. The Cluster uses consistent hashing for even data distribution across nodes, while Redis relies on a modulo hashing algorithm.
How does Redis Cluster work?
Well, Redis Cluster operates by distributing data across multiple nodes within a cluster. Nodes store specific subsets of data, with replication across multiple nodes to ensure redundancy and reliability in Redis Cluster. Hence, it offers automatic failover, enabling data recovery from other nodes in the cluster in the event of a node failure. Furthermore, it supports automatic sharding, facilitating the distribution of data across multiple nodes for improved performance and scalability.
Advantages of Redis Cluster
Here, Redis Cluster offers several advantages for distributed data storage:
Scalability
Well, it exhibits exceptional scalability, allowing effortless adjustment to the requirements of an application. With the ability to accommodate hundreds of nodes, it effectively handles substantial data volumes. Automatic data sharding ensures the even distribution of data across multiple nodes, enabling applications to access data rapidly and efficiently. Automatic failover guarantees data availability even in the presence of node failures.
Performance
It leverages an in memory data structure store, resulting in high performance data storage and retrieval. Storing data in memory enables rapid and efficient access by applications. Automatic data sharding ensures even distribution of data across nodes, enhancing data access efficiency. Automatic failover ensures uninterrupted availability of data, even in the face of node failures.
Security
What is more, it provides a secure environment for data storage and retrieval. It employs authentication mechanisms to restrict access to authorized users only. Encryption safeguards data during storage and transmission by utilizing techniques that ensure confidentiality and prevent unauthorized access. Access control mechanisms further enhance security, limiting data access to authorized individuals.
Fault Tolerance
With data replication across multiple nodes, it offers fault tolerance. In case of node failures, data is retrieved from other nodes, ensuring data reliability and minimizing disruptions.
Limitations of Redis Cluster
While there is many benefits, it is important to be aware of its limitations:
- Complexity: Setting up and managing the Cluster is more complex compared to standalone Redis. The distributed nature of the cluster requires additional configuration and monitoring.
- Cost: Redis it demands more hardware resources than standalone Redis due to the distributed nature of the cluster. Deploying and maintaining a cluster with multiple nodes entails higher infrastructure costs.
- Availability: it does not provide 100% availability. In scenarios where a majority of nodes fail, the cluster becomes unavailable. While automatic failover mitigates the impact of individual node failures, a significant number of simultaneous node failures impacts cluster availability.
- Eventual Consistency: adopts an eventual consistency model, which means that updates to the data may take some time to propagate across all nodes in the cluster. This delay results in temporary inconsistencies in data retrieval until the updates are fully propagated.
Redis Cluster: How to Build a Scalable Redis Cache Infrastructure
Follow the article to we show you how to set up a four node the cluster and scale it by adding one more node.
Prerequisites
- Four servers running Ubuntu operating systems.
- A root user or a user with sudo privileges.
For the purpose of this article, we use the following setup:
Getting Started
Before starting, it is recommended to update all servers using the following command.
apt update -y
apt upgrade -y
Once all servers are updated, edit the sysctl.conf file and tweak some settings.
nano /etc/sysctl.conf
Add the following lines.
net.core.somaxconn=65535
vm.overcommit_memory=1
Save and close the file when you are finished.
Install Redis Server On All Nodes
First, you need to install the Redis server package on all nodes. Install it using the following command.
apt install redis-server
After installing the Redis server, stop the Redis service and disable it.
systemctl stop redis-server
systemctl disable redis-server
Configure Redis Server on All Nodes
First, create some required directories for Redis on all nodes.
mkdir /etc/redis/cluster
mkdir /etc/redis/cluster/7000
mkdir /var/lib/redis/7000
mkdir /etc/redis/cluster/7001
mkdir /var/lib/redis/7001
Next, create a Redis master configuration file.
nano /etc/redis/cluster/7000/redis_7000.conf
Add the following configurations.
port 7000
dir /var/lib/redis/7000/
appendonly no
protected-mode no
cluster-enabled yes
cluster-node-timeout 5000
cluster-config-file /etc/redis/cluster/7000/nodes_7000.conf
pidfile /var/run/redis/redis_7000.pid
logfile /var/log/redis/redis_7000.log
loglevel notice
requirepass [YOUR_SECURE_ACCESS_KEY]
masterauth [YOUR_SECURE_ACCESS_KEY]
Save and close the file then create a replica configuration file.
nano /etc/redis/cluster/7001/redis_7001.conf
Add the following configurations.
port 7001
dir /var/lib/redis/7001
appendonly no
protected-mode no
cluster-enabled yes
cluster-node-timeout 5000
cluster-config-file /etc/redis/cluster/7001/nodes_7001.conf
pidfile /var/run/redis/redis_7001.pid
logfile /var/log/redis/redis_7001.log
loglevel notice
masterauth [YOUR_SECURE_ACCESS_KEY]
requirepass [YOUR_SECURE_ACCESS_KEY]
Save and close the file then set proper permission and ownership using the following command.
chown redis:redis -R /var/lib/redis
chmod 770 -R /var/lib/redis
chown redis:redis -R /etc/redis
Create a Redis Service File on All Nodes
Next, you need to create a systemd service file for Redis on all Nodes. First, create a master service file.
nano /etc/systemd/system/redis_7000.service
Add the following configurations.
[Unit]
Description=Redis key-value database on 7000
After=network.target
[Service]
ExecStart=/usr/bin/redis-server /etc/redis/cluster/7000/redis_7000.conf --supervised systemd
ExecStop=/bin/redis-cli -h 127.0.0.1 -p 7000 shutdown
Type=notify
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=0755
LimitNOFILE=65535
[Install]
WantedBy=multi-user.target
Save and close the file then create a replica service file.
nano /etc/systemd/system/redis_7001.service
Add the following configurations.
[Unit]
Description=Redis key-value database on 7001
After=network.target
[Service]
ExecStart=/usr/bin/redis-server /etc/redis/cluster/7001/redis_7001.conf --supervised systemd
ExecStop=/bin/redis-cli -h 127.0.0.1 -p 7001 shutdown
Type=notify
User=redis
Group=redis
RuntimeDirectory=/etc/redis/cluster/7001
RuntimeDirectoryMode=0755
LimitNOFILE=65535
[Install]
WantedBy=multi-user.target
Save and close the file then reload the systemd daemon to apply the changes.
systemctl daemon-reload
Next, enable both services to start at system reboot.
systemctl enable redis_7000.service
systemctl enable redis_7001.service
Next, restart all nodes to apply the changes.
reboot
Verify Redis Installation
At this point, the Redis server is installed and configured on all servers. Now, you need to verify whether Redis runs or not.
Login to Node1 and check the Redis log using the following command.
tail -f /var/log/redis/redis_7000.log
If everything is fine, you will see the following screen.
Also verify the Redis replica using the following command.
tail -f /var/log/redis/redis_7001.log
See the following screen.
To check the status of both Redis services, run the following command.
systemctl status redis_7000.service redis_7001.service
See the Redis status on the following screen.
Setting Up Redis Cluster
In this section, we configure the Redis cluster using the three nodes. Login to Node1 and run the following command to create it.
redis-cli -c -h 192.168.100.1 -p 7000 -a [YOUR_SECURE_ACCESS_KEY] --cluster create 192.168.100.1:7000 192.168.100.2:7000 192.168.100.3:7000 192.168.100.1:7001 192.168.100.2:7001 192.168.100.3:7001 --cluster-replicas 1
You see the following screen.
The above command creates the cluster and divide 16384 hash slots among three servers.
Now, log in to the Redis node using the following command.
redis-cli -c -h 192.168.100.1 -p 7000 -a [YOUR_SECURE_ACCESS_KEY]
Once you are connected to the Redis shell, verify the status using the following command.
192.168.100.1:7000> CLUSTER NODES
If everything is fine, you see the following screen.
Now exit from the Redis shell using the following command.
192.168.100.1:7000> exit
Validate Redis Cluster Functionality
At this point, the cluster is configured. Now, you need to run the SET and GET commands to check the behaviour of Redis:
On Node1, connect to the Redis using the following command.
redis-cli -c -h 192.168.100.1 -p 7000 -a [YOUR_SECURE_ACCESS_KEY]
Now, run the following commands one by one to check the cluster.
192.168.100.1:7000> set a 1
-> Redirected to slot [15495] located at 192.168.100.3:7000
OK
192.168.100.3:7000> set b 2
-> Redirected to slot [3300] located at 192.168.100.1:7000
OK
192.168.100.1:7000> set c 3
-> Redirected to slot [7365] located at 192.168.100.2:7000
OK
192.168.100.2:7000> set d 4
-> Redirected to slot [11298] located at 192.168.100.3:7000
OK
192.168.100.3:7000> get b
-> Redirected to slot [3300] located at 192.168.100.1:7000
"2"
192.168.100.1:7000> get a
-> Redirected to slot [15495] located at 192.168.100.3:7000
"1"
192.168.100.3:7000> get c
-> Redirected to slot [7365] located at 192.168.100.2:7000
"3"
192.168.100.2:7000> get d
-> Redirected to slot [11298] located at 192.168.100.3:7000
"4"
As you see that Redis sends the keys to the correct hash slots in each server.
Test Redis Cluster Failover
In this section, we stop the master process on Node1 and verify whether the Redis cluster promotes other servers to master or not.
On Node1, run the following command to stop the Redis process for 400 seconds.
redis-cli -c -h 192.168.100.1 -p 7000 -a [YOUR_SECURE_ACCESS_KEY] DEBUG sleep 400
This command stops Redis on Node1 and force Replica process 7001 on Node2 to take over and change its role to Master.
Now, log in to Node2 and connect to the Redis Replica process.
redis-cli -c -h 192.168.100.2 -p 7001 -a [YOUR_SECURE_ACCESS_KEY]
Once you are logged in, run the following command.
192.168.100.2:7001> info
You should see that the Replica process running in Server 2 got promoted to Master.
Scale Redis Cluster
Besides, Redis also allows you to scale the cluster by adding one or more nodes to the existing cluster. In this section, we the add Node4. .
On Node4, run the following command to add the Master process of the current new server to the cluster:
redis-cli -c -h 192.168.100.4 -p 7000 -a [YOUR_SECURE_ACCESS_KEY] --cluster add-node 192.168.100.4:7000 192.168.100.1:7000
If everything is fine, see the following screen.
Next, Reshard the cluster to distribute the key slots among all the nodes.
On Master Node, run the following command to Reshard the cluster.
redis-cli -c -h 192.168.100.1 -p 7000 -a [YOUR_SECURE_ACCESS_KEY] --cluster reshard 192.168.100.4:7000
This redistributes the key slots among all nodes. Now, connect to the Redis using the following command.
redis-cli -c -h 192.168.100.1 -p 7000 -a [YOUR_SECURE_ACCESS_KEY]
Once you are connected, verify the cluster status using the following command.
192.168.100.1:7000> CLUSTER NODES
You should see all of your added nodes on the following screen.
Thank you for reading Redis Cluster: How to Build a Scalable Redis Cache Infrastructure. We shall conclude.
Redis Cluster: How to Build a Scalable Redis Cache Infrastructure Conclusion
In this guide, we showed you how to install the Redis server on Ubuntu Linux. Then, we demonstrated setting up three node Redis cluster and verified its failover behaviours. We have also explained how to scale the Redis cluster by adding one more node. I hope you can now use this setup on the production environment to deploy the fully functional Redis cluster.
Finally, Redis Cluster is a powerful tool for scaling and enhancing the reliability of Redis installations. However, it’s crucial to consider its limitations before adoption. For simple and reliable in-memory database needs, it is recommended to use Redis. On the other hand, if scalability and reliability are priorities, Redis Cluster is a suitable choice. Choose wisely based on your requirements.
