Redis3.2.5 集群搭建

Posted 2020-10-01 YSHY

1：集群中的机器信息

IP　　　　　　　　PORT

192.168.3.10　　　7000,7001,7002

192.168.3.11　　　7004,7005,7006

2：安装Redis

分别在10与11机器上面安装Redis。

（1）解压安装

#tar -zxvf redis-3.2.5.tar.gz

#cd redis-3.2.5

#make && make install

（2）修改系统的限制文件

修改文件/etc/security/limits.conf，加入以下两行：

* soft nofile 102400

* hard nofile 102400

3：创建Redis配置文件

【192.168.3.10】

wzh@hd-master:/usr/local/redis_cluster$

创建7000/redis.conf，7001/redis.conf，7002/redis.conf

【192.168.3.11】

wzh@hd-master:/usr/local/redis_cluster$

创建7004/redis.conf，7005/redis.conf，7006/redis.conf

修改配置文件：

requirepass redis #去掉行前的注释，并修改密码为所需的密码,保存文件

daemonize yes  # 是否后台运行

port 7000 # 端口

cluster-enabled yes #开启集群

cluster-config-file nodes-7000.conf

cluster-node-timeout 5000

appendonly no

protected-mode no

# bind 127.0.0.1

说明：

cluster-enabled 选项用于开实例的集群模式， 而 cluster-conf-file 选项则设定了保存节点配置文件的路径， 默认值为nodes.conf 。其他参数相信童鞋们都知道。节点配置文件无须人为修改， 它由 Redis 集群在启动时创建， 并在有需要时自动进行更新。

 

4：启动Redis服务

【192.168.3.10】

wzh@hd-master:~$ redis-server /usr/local/redis_cluster/7000/redis.conf

wzh@hd-master:~$ redis-server /usr/local/redis_cluster/7001/redis.conf

wzh@hd-master:~$ redis-server /usr/local/redis_cluster/7002/redis.conf

【192.168.3.11】

wzh@hd-slave1:~$ redis-server /usr/local/redis_cluster/7004/redis.conf

wzh@hd-slave1:~$ redis-server /usr/local/redis_cluster/7005/redis.conf

wzh@hd-slave1:~$ redis-server /usr/local/redis_cluster/7006/redis.conf

5：集群设置

（1）安装ruby

$tar zxvf ruby-2.3.1.tar.gz

$ ./configure

$ make

$ sudo make install

（2）安装rubygems

wzh@hd-master:~$ tar zxvf rubygems-2.6.8.tgz

root@hd-master:~/rubygems-2.6.8# ruby setup.rb

RubyGems 2.6.8 installed

Parsing documentation for rubygems-2.6.8

Installing ri documentation for rubygems-2.6.8

（3）安装 redis-3.3.1.gem

wzh@hd-master:~$ gem install -l redis-3.3.1.gem

ERROR:  Loading command: install (LoadError)

        cannot load such file -- zlib

ERROR:  While executing gem ... (NoMethodError)

    undefined method `invoke_with_build_args\' for nil:NilClass

手动安装zlib包解决，进入 ruby源码

wzh@hd-master:~/ruby-2.3.1/ext/zlib$ sudo ruby extconf.rb

root@hd-master:~/ruby-2.3.1/ext/zlib# make

compiling zlib.c

linking shared-object zlib.so

root@hd-master:~/ruby-2.3.1/ext/zlib# make install

/usr/bin/install -c -m 0755 zlib.so /usr/local/lib/ruby/site_ruby/2.3.0/x86_64-linux

 

再次安装：

root@hd-master:~#  gem install -l redis-3.3.1.gem

Successfully installed redis-3.3.1

Parsing documentation for redis-3.3.1

Installing ri documentation for redis-3.3.1

Done installing documentation for redis after 0 seconds

1 gem installed

root@hd-master:~#

（4）创建集群

redis-trib  create  --replicas  1  192.168.3.10:7000 192.168.3.10:7001  192.168.3.10:7002  192.168.3.11:7004  192.168.3.11:7005  192.168.3.11:7006

 

root@hd-master:/usr/local/redis_cluster# redis-trib  create  --replicas  1  192.168.3.10:7000 192.168.3.10:7001  192.168.3.10:7002 192.168.3.11:7004  192.168.3.11:7005  192.168.3.11:7006

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

192.168.3.10:7000

192.168.3.11:7004

192.168.3.10:7001

Adding replica 192.168.3.11:7005 to 192.168.3.10:7000

Adding replica 192.168.3.10:7002 to 192.168.3.11:7004

Adding replica 192.168.3.11:7006 to 192.168.3.10:7001

M: 4807c76cebb29f5a96e607197fd760193dea06ef 192.168.3.10:7000

   slots:0-5460 (5461 slots) master

M: edca0a2ad7371e7c4af09fbb5457c8ed6ff95a3f 192.168.3.10:7001

   slots:10923-16383 (5461 slots) master

S: ee23aaa752bf60c748fd5a9814560a4742f04d0c 192.168.3.10:7002

   replicates ea3a96fefe14dcb9d0e6b9f3e0345d5b4ea8871a

M: ea3a96fefe14dcb9d0e6b9f3e0345d5b4ea8871a 192.168.3.11:7004

   slots:5461-10922 (5462 slots) master

S: e917ef93693360c9648a254c1c95a88b5da9cdd0 192.168.3.11:7005

   replicates 4807c76cebb29f5a96e607197fd760193dea06ef

S: b04f5c57362bcabfba5c17c1260ada1a76aa1d50 192.168.3.11:7006

   replicates edca0a2ad7371e7c4af09fbb5457c8ed6ff95a3f

Can I set the above configuration? (type \'yes\' to accept): yes

/usr/local/lib/ruby/gems/2.3.0/gems/redis-3.0.0/lib/redis/client.rb:79:in `call\': ERR Slot 0 is already busy (Redis::CommandError)

        from /usr/local/lib/ruby/gems/2.3.0/gems/redis-3.0.0/lib/redis.rb:2190:in `block in method_missing\'

        from /usr/local/lib/ruby/gems/2.3.0/gems/redis-3.0.0/lib/redis.rb:36:in `block in synchronize\'

        from /usr/local/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize\'

        from /usr/local/lib/ruby/gems/2.3.0/gems/redis-3.0.0/lib/redis.rb:36:in `synchronize\'

        from /usr/local/lib/ruby/gems/2.3.0/gems/redis-3.0.0/lib/redis.rb:2189:in `method_missing\'

        from /usr/local/bin/redis-trib:212:in `flush_node_config\'

        from /usr/local/bin/redis-trib:776:in `block in flush_nodes_config\'

        from /usr/local/bin/redis-trib:775:in `each\'

        from /usr/local/bin/redis-trib:775:in `flush_nodes_config\'

        from /usr/local/bin/redis-trib:1296:in `create_cluster_cmd\'

        from /usr/local/bin/redis-trib:1701:in `<main>\'

root@hd-master:~#

【问题解决】

根据网上其他资料，将redis-3.3.2.gem替换为redis-3.0.0.gem

root@hd-master:/usr/local/redis_cluster# gem list --local

 

*** LOCAL GEMS ***

 

bigdecimal (default: 1.2.8)

did_you_mean (1.0.0)

io-console (default: 0.4.5)

json (default: 1.8.3)

minitest (5.8.3)

net-telnet (0.1.1)

power_assert (0.2.6)

psych (default: 2.0.17)

rake (10.4.2)

rdoc (default: 4.2.1)

redis (3.3.2)

test-unit (3.1.5)

root@hd-master:/usr/local/redis_cluster# gem uninstall redis --version 3.3.2

Successfully uninstalled redis-3.3.2

root@hd-master:/usr/local/redis_cluster#

 

 

root@hd-master:~# gem install --local redis-3.0.0.gem

Successfully installed redis-3.0.0

Parsing documentation for redis-3.0.0

Installing ri documentation for redis-3.0.0

Done installing documentation for redis after 0 seconds

1 gem installed

 

将redis-3.3.2.gem替换为redis-3.0.0.gem之后启动还是报错经检查，这是由于上一次配置集群失败时留下的配置信息导致的。 只要把redis.conf中定义的 cluster-config-file 所在的文件删除，重新启动redis-server及运行redis-trib即可。

分别在10与11机器上面删除nodes-*.conf配置文件

root@hd-master:/usr/local/redis_cluster#

-rw-r--r--  1 root root  751  8月  4 17:02 nodes-7000.conf

-rw-r--r--  1 root root  751  8月  4 17:02 nodes-7001.conf

-rw-r--r--  1 root root  751  8月  4 17:02 nodes-7002.conf

 

root@hd-master:/usr/local/redis_cluster# redis-trib  create  --replicas  1  192.168.3.10:7000 192.168.3.10:7001  192.168.3.10:7002  192.168.3.11:7004  192.168.3.11:7005  192.168.3.11:7006

>>> Performing Cluster Check (using node 192.168.3.10:7000)

M: 5b5cdec3e1b7ce29a966034a610e937cfb3f83d1 192.168.3.10:7000

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: 044ea73e14d77b7a19e2d5c3c5ca7646deb3c15d 192.168.3.10:7001

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: 57ceff995451b80eebf90996f23b92f0446ed034 192.168.3.10:7002

   slots: (0 slots) slave

   replicates ed779d661623c2c3eb44db98260948dedcc8f702

M: ed779d661623c2c3eb44db98260948dedcc8f702 192.168.3.11:7004

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

S: 367dbb4888a0123f205d16f4d720ab703807b7b6 192.168.3.11:7005

   slots: (0 slots) slave

   replicates 5b5cdec3e1b7ce29a966034a610e937cfb3f83d1

S: b8d273e16dca3110488d80166351dec3d0833d32 192.168.3.11:7006

   slots: (0 slots) slave

   replicates 044ea73e14d77b7a19e2d5c3c5ca7646deb3c15d

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

6：集群测试

【192.168.3.10】

wzh@hd-master:~$ redis-cli -c -p 7000

192.168.3.10:7000> set name test

-> Redirected to slot [5798] located at 192.168.3.11:7004

OK

192.168.3.11:7004> get name

"test"

192.168.3.11:7004>

 

【192.168.3.11】

wzh@hd-slave1:~$ redis-cli -c -p 7004

127.0.0.1:7004> get name

"test"

127.0.0.1:7004> get address

-> Redirected to slot [3680] located at 192.168.3.10:7000

"qingdao"

192.168.3.10:7000>

 

7：Spring-boot测试

（1）build.gradle增加以下配置

dependencies {
    compile(\'org.springframework.boot:spring-boot-starter-data-redis\')
    compile(\'org.springframework.boot:spring-boot-starter-web\')
    testCompile(\'org.springframework.boot:spring-boot-starter-test\')
}

 

（2）application.properties

spring.redis.cluster.nodes=192.168.3.10:7000,192.168.3.10:7001,192.168.3.10:7002,192.168.3.11:7004,192.168.3.11:7005,192.168.3.11:7006

 

（3）TestCtrl.java

package com.example.springbootredis.test;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.ValueOperations;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import java.util.HashMap;
import java.util.Map;

/**
 * Created by yan on 2017/8/6.
 */
@RestController
@RequestMapping(
        value = "/api/test",
        produces = "application/json;charset=utf-8",
        headers = "Accept=application/json")
public class TestCtrl {
    @Autowired
    RedisTemplate<String, String> redisTemplate;

    @RequestMapping(
            value = "/set/{key}/{value}",
            method = RequestMethod.GET
    )
    public Map<String, String>  set(@PathVariable String key, @PathVariable String value){
        ValueOperations<String,String> valueOperations = redisTemplate.opsForValue();

        valueOperations.set(key, value);

        Map<String, String> map = new HashMap<String, String>();
        map.put(key, value);

        return map;
    }

    @RequestMapping(
            value = "/get/{key}",
            method = RequestMethod.GET
    )
    public Map<String, String>  get(@PathVariable String key){
        ValueOperations<String,String> valueOperations = redisTemplate.opsForValue();

        String value = valueOperations.get(key);

        Map<String, String> map = new HashMap<String, String>();
        map.put(key, value);

        return map;
    }
}

 

 

8：参考

http://www.linuxidc.com/Linux/2016-09/135122.htm

http://www.cnblogs.com/yuanermen/p/5717885.html