c4rt1y

redis_cluster集群搭建

0x01 redis cluster介绍

redis3.0之后,推出了Cluster,可以进行集群化管理,花了点时间,进行测试模拟。
在官方文档Cluster Spec中,作者详细介绍了Redis集群为什么要设计成现在的样子。最核心的目标有三个:

性能:这是Redis赖以生存的看家本领,增加集群功能后当然不能对性能产生太大影响,所以Redis采取了P2P而非Proxy方式、异步复制、客户端重定向等设计,而牺牲了部分的一致性、使用性。
水平扩展:集群的最重要能力当然是扩展,文档中称可以线性扩展到1000结点。
可用性:在Cluster推出之前,可用性要靠Sentinel保证。有了集群之后也自动具有了Sentinel的监控和自动Failover能力。

0x02 环境介绍

#一共2台服务器
master.com 10.10.10.20
node01.com 10.10.10.30

#关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service

#两种方法设置host(每台对应每台的host地址)
echo "master.com" > /etc/hostname
hostname master.com

#设置host
cat >>/etc/hosts <<EOF 
10.10.10.20 master.com 
10.10.10.30 node01.com
EOF	

#关闭selinux,需要重启
sed -i 's:SELINUX=enforcing:SELINUX=disabled:g' /etc/selinux/config

#临时关闭seLinux
setenforce 0

#重启
reboot

0x03 环境安装

##两台服务器上同时安装

#安装依赖环境
yum install gcc zlib-devel openssl openssl-devel -y

#安装ruby,因为redis-trib.rb是ruby爱好者写的
wget https://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.7.tar.gz && tar zxf ruby-2.2.7.tar.gz && cd ruby-2.2.7 &&./configure  --prefix=/usr/local/ruby && make && make install

#设置环境变量
echo "PATH=$PATH:/usr/local/ruby/bin;export PATH" >> /etc/profile
source /etc/profile

#测试
ruby -version

#安装ruby的redis模块
gem install redis 

#下载redis
wget http://download.redis.io/releases/redis-3.2.4.tar.gz
#解压redis
tar zxf redis-3.2.4.tar.gz 
#编译安装
cd redis-3.2.4 && make && make install

#拷贝文件
/bin/cp /root/redis-3.2.4/src/redis-trib.rb  /usr/local/bin/


##在master上创建集群
#创建redis集群目录
mkdir -p /root/redis-3.2.4/redis_cluster/{7001,7002,7003}

#将redis的集群目录分别拷贝redis.conf配置文件,这里我才去了偷懒模式,一句话搞定
echo  /root/redis-3.2.4/redis_cluster/7001/ /root/redis-3.2.4/redis_cluster/7002/ /root/redis-3.2.4/redis_cluster/7003/  | xargs -n 1 cp -v /root/redis-3.2.4/redis.conf 

#等同于三条语句
cp  /root/redis-3.2.4/redis.conf   /root/redis-3.2.4/redis_cluster/7001/
cp  /root/redis-3.2.4/redis.conf   /root/redis-3.2.4/redis_cluster/7002/
cp  /root/redis-3.2.4/redis.conf   /root/redis-3.2.4/redis_cluster/7003/


##在node01上创建集群
#创建redis集群目录
mkdir -p /root/redis-3.2.4/redis_cluster/{7004,7005,7006}

#将redis的集群目录分别拷贝redis.conf配置文件
echo  /root/redis-3.2.4/redis_cluster/7004/ /root/redis-3.2.4/redis_cluster/7005/ /root/redis-3.2.4/redis_cluster/7006/  | xargs -n 1 cp -v /root/redis-3.2.4/redis.conf 


##现在修改两边的配置文件
bind 10.10.10.20  	#监听IP必须修改,目前是10.10.10.20这台服务器
daemonize yes 		#后台自启
pidfile /var/run/redis_7001.pid  #pidfile文件
port 7001 	#端口,这里注意我现在配置的是master上的/root/redis-3.2.4/redis_cluster/7001/redis.conf文件
cluster-enabled yes  	#开启集群  
cluster-config-file nodes-7001.conf   #集群的配置  配置文件首次启动自动生成 7000
cluster-node-timeout 15000  	#延迟时间
cluster-require-full-coverage yes  	#redis集群有16383个slot组成,如果出现宕机,直接全部挂了,这个是为了防止效果
appendonly yes 			#日志记录
logfile "/var/log/redis/redis.log"   #也可以使用这个日志记录  


## master上设置

#如果开启了logfile,需要创建文件夹
mkdir /var/log/redis
touch /var/log/redis/redis.log
#修改配置文件
#7001.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.20/g' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7001.pid/#'  /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i 's/port 6379/port 7001/g' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7001.conf' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i '/# cluster-require-full-coverage yes/a\cluster-require-full-coverage yes' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7001/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7001/redis.conf
#7002.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.20/g' /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7002.pid/#'  /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i 's/port 6379/port 7002/g' /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7002.conf' /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7002/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7002/redis.conf
#7003.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.20/g' /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7003.pid/#'  /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i 's/port 6379/port 7003/g' /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7003.conf' /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7003/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7003/redis.conf

#启动redis
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7001/redis.conf
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7002/redis.conf
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7003/redis.conf

## node01上设置

#如果开启了logfile,需要创建文件夹
mkdir /var/log/redis
touch /var/log/redis/redis.log

#修改配置文件
#7004.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.30/g' /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7004.pid/#'  /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i 's/port 6379/port 7004/g' /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7004.conf' /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7004/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7004/redis.conf
#7005.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.30/g' /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7005.pid/#'  /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i 's/port 6379/port 7005/g' /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7005.conf' /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7005/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7005/redis.conf
#7006.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.30/g' /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7006.pid/#'  /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i 's/port 6379/port 7006/g' /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7006.conf' /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7006/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7006/redis.conf


#启动redis
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7004/redis.conf
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7005/redis.conf
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7006/redis.conf


##在master上进行执行操作,做集群处理
/root/redis-3.2.4/src/redis-trib.rb  create  --replicas  1  10.10.10.20:7001  10.10.10.20:7002 10.10.10.20:7003 10.10.10.30:7004 10.10.10.30:7005 10.10.10.30:7006

#测试
/root/redis-3.2.4/src/redis-cli -h 10.10.10.20 -p 7001 

#不断写入以下数据,她会进行分散存储
set foo1 1
set foo2 2

0x04 redis cluster命令介绍

#查看帮助信息(直接回车)
/root/redis-3.2.4/src/redis-trib.rb

#查看参数
create 	#创建集群
check   #检查集群
fix   	#修复集群
reshard 	#交互迁移
add-node 	#增加节点
del-node 	#删除节点
set-timeout	#设置延迟

##演示1
#增加节点

#master上
mkdir /root/redis-3.2.4/redis_cluster/7010
#复制配置文件
cp  /root/redis-3.2.4/redis.conf   /root/redis-3.2.4/redis_cluster/7010/
#7010.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.20/g' /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7010.pid/#'  /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i 's/port 6379/port 7010/g' /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7010.conf' /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7010/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7010/redis.conf
#启动
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7010/redis.conf


##node01上
mkdir /root/redis-3.2.4/redis_cluster/7021
#复制配置文件
cp  /root/redis-3.2.4/redis.conf   /root/redis-3.2.4/redis_cluster/7021/
#7021.conf配置文件
sed -i 's/^bind 127.0.0.1/bind 10.10.10.30/g' /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i 's/daemonize no/daemonize yes/g' /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i 's#pidfile /var/run/redis_6379.pid#pidfile /var/run/redis_7021.pid/#'  /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i 's/port 6379/port 7021/g' /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i '/# cluster-enabled yes/a\cluster-enabled yes' /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i '/# cluster-config-file nodes-6379.conf/a\cluster-config-file nodes-7021.conf' /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i '/# cluster-node-timeout 15000/a\cluster-node-timeout 5000' /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i 's/appendonly no/appendonly yes/g' /root/redis-3.2.4/redis_cluster/7021/redis.conf
sed -i 's#logfile ""#logfile "/var/log/redis/redis.log"#'  /root/redis-3.2.4/redis_cluster/7021/redis.conf
#启动
/root/redis-3.2.4/src/redis-server /root/redis-3.2.4/redis_cluster/7021/redis.conf

#添加节点(一个作为Master,一个作为其Slave)
/root/redis-3.2.4/src/redis-trib.rb add-node 10.10.10.20:7010 10.10.10.30:7021

#删除节点
/root/redis-3.2.4/src/redis-trib.rb del-node 10.10.10.20:7010 10.10.10.30:7021

#数据迁移,讲节点数据迁移到7010上
/root/redis-3.2.4/src/redis-trib.rb reshard 10.10.10.20:7010

0x05 资料来源

https://redis.io/topics/cluster-tutorial
http://blog.163.com/mity_rui@126/blog/static/10981361820171235252431/
http://www.cnblogs.com/yuanermen/p/5717885.html
http://blog.csdn.net/dc_726/article/details/48552531
GoTop