lvs-dr模式部署遇到问题，求教老鸟

Posted 2020-10-24

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了lvs-dr模式部署遇到问题，求教老鸟相关的知识，希望对你有一定的参考价值。

lvs 实验

部署LVS-DR模式遇到了问题，特向老鸟们请教

在Client发起请求时Director和Real Server 都出现SYN-RECV 的状态，感觉是我的Real Server没有响应Director发来的请求。

请老鸟们帮我排查一下问题，万分感谢！

拓扑以及IP地址规划

1、打开GATEWAY路由转发功能技术分享图片

[[email protected] ~]# sysctl -p

net.ipv4.ip_forward = 1

#添加源地址转换

[[email protected] ~]# iptables -t nat -A POSTROUTING -s 192.168.3.0/24 ! -d 192.168.3.0/24 -j SNAT --to-source 10.10.3.139

#检查结果

[[email protected] ~]# service iptables status

Table: nat

Chain PREROUTING (policy ACCEPT)

num target prot opt source destination

Chain POSTROUTING (policy ACCEPT)

num target prot opt source destination

1 SNAT all -- 192.168.3.0/24 !192.168.3.0/24 to:10.10.3.139

2、配置RIP并将网关指向GATWEAY（192.168.3.139）

#检查结果ping Client 地址

[[email protected]ode136 ~]# ping 10.10.3.100

PING 10.10.3.100 (10.10.3.100) 56(84) bytes of data.

64 bytes from 10.10.3.100: icmp_seq=1 ttl=127 time=0.769 ms

64 bytes from 10.10.3.100: icmp_seq=2 ttl=127 time=0.578 ms

64 bytes from 10.10.3.100: icmp_seq=3 ttl=127 time=0.576 ms

64 bytes from 10.10.3.100: icmp_seq=4 ttl=127 time=6.18 ms

[[email protected] ~]# ping 10.10.3.100

PING 10.10.3.100 (10.10.3.100) 56(84) bytes of data.

64 bytes from 10.10.3.100: icmp_seq=1 ttl=127 time=0.590 ms

64 bytes from 10.10.3.100: icmp_seq=2 ttl=127 time=0.484 ms

64 bytes from 10.10.3.100: icmp_seq=3 ttl=127 time=0.843 ms

64 bytes from 10.10.3.100: icmp_seq=4 ttl=127 time=0.597 ms

3、配置Real Server 的ARP抑制

echo 2 > /proc/sys/net/ipv4/conf/eth0/arp_announce

echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

echo 1 > /proc/sys/net/ipv4/conf/eth0/arp_ignore

echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

#检查结果

[[email protected] ~]# sysctl -a |grep arp_ignor

net.ipv4.conf.all.arp_ignore = 1

net.ipv4.conf.default.arp_ignore = 0

net.ipv4.conf.lo.arp_ignore = 0

net.ipv4.conf.eth0.arp_ignore = 1

[[email protected] ~]# sysctl -a |grep arp_announce

net.ipv4.conf.all.arp_announce = 2

net.ipv4.conf.default.arp_announce = 0

net.ipv4.conf.lo.arp_announce = 0

net.ipv4.conf.eth0.arp_announce = 2

4、打开Director的路由转发，配置DIP（不指DIP的网关）

[[email protected] ~]# sysctl -p

net.ipv4.ip_forward = 1

[[email protected] ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0

1 DEVICE=eth0

2 TYPE=Ethernet

3 UUID=d1be5ee5-76b8-4dfb-a997-67b3ca4a82d1

4 ONBOOT=yes

5 NM_CONTROLLED=yes

6 BOOTPROTO=none

7 HWADDR=00:0C:29:D4:EE:E2

8 IPADDR=192.168.3.135

9 PREFIX=24

10 DEFROUTE=yes

11 IPV4_FAILURE_FATAL=yes

12 IPV6INIT=no

13 NAME="System eth0"

#检查结果与Real Server 通讯

[[email protected] ~]# ping 192.168.3.136

PING 192.168.3.136 (192.168.3.136) 56(84) bytes of data.

64 bytes from 192.168.3.136: icmp_seq=1 ttl=64 time=1.44 ms

64 bytes from 192.168.3.136: icmp_seq=2 ttl=64 time=0.099 ms

[[email protected] ~]# ping 192.168.3.137

PING 192.168.3.137 (192.168.3.137) 56(84) bytes of data.

64 bytes from 192.168.3.137: icmp_seq=1 ttl=64 time=1.30 ms

64 bytes from 192.168.3.137: icmp_seq=2 ttl=64 time=0.190 ms

5、配置VIP

#配置Director上的VIP

[[email protected] ~]# ifconfig eth0:0 10.10.3.135/16 up

#检查结果

[[email protected] ~]# ifconfig

eth0 Link encap:Ethernet HWaddr 00:0C:29:D4:EE:E2

inet addr:192.168.3.135 Bcast:192.168.3.255 Mask:255.255.255.0

inet6 addr: fe80::20c:29ff:fed4:eee2/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:920681 errors:0 dropped:0 overruns:0 frame:0

TX packets:1071 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:60442376 (57.6 MiB) TX bytes:91252 (89.1 KiB)

eth0:0 Link encap:Ethernet HWaddr 00:0C:29:D4:EE:E2

inet addr:10.10.3.135 Bcast:10.10.3.255 Mask:255.255.255.0

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:65536 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

#配置Real Server 上的VIP

[[email protected] ~]# ifconfig lo:0 10.10.3.135 dev lo:0 up

[[email protected] ~]# ifconfig

eth0 Link encap:Ethernet HWaddr 00:0C:29:2F:62:37

inet addr:192.168.3.136 Bcast:192.168.3.255 Mask:255.255.255.0

inet6 addr: fe80::20c:29ff:fe2f:6237/64 Scope:Link

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

RX packets:771221 errors:0 dropped:0 overruns:0 frame:0

TX packets:2589 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:50744484 (48.3 MiB) TX bytes:139999 (136.7 KiB)

lo Link encap:Local Loopback

inet addr:127.0.0.1 Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING MTU:65536 Metric:1

RX packets:0 errors:0 dropped:0 overruns:0 frame:0

TX packets:0 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

lo:0 Link encap:Local Loopback

inet addr:10.10.3.135 Mask:0.0.0.0

UP LOOPBACK RUNNING MTU:65536 Metric:1

#添加请求流量引导路由

[[email protected] ~]# route add -host 10.10.3.135 dev eth0:0

#检查结果

[[email protected] ~]# route -n

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

10.10.3.135 0.0.0.0 255.255.255.255 UH 0 0 0 eth0

192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

10.10.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0

[[email protected] ~]# route add -host 10.10.3.135 dev lo:0

[[email protected] ~]# route -n

Kernel IP routing table

Destination Gateway Genmask Flags Metric Ref Use Iface

10.10.3.135 0.0.0.0 255.255.255.255 UH 0 0 0 lo

192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0

169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0

0.0.0.0 192.168.3.139 0.0.0.0 UG 0 0 0 eth0

#检查从客户端到VIP的通讯

C:\Users\Administrator>ping 10.10.3.135

正在 Ping 10.10.3.135 具有 32 字节的数据:

来自 10.10.3.135 的回复: 字节=32 时间<1ms TTL=64

6、配置lvs

[[email protected] ~]# ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port Forward Weight ActiveConn InActConn

TCP 10.10.3.135:80 rr

-> 192.168.3.136:80 Route 1 0 0

-> 192.168.3.137:80 Route 1 0 0

7、发起访问时lvs状态

[[email protected] ~]# ipvsadm -lnc

IPVS connection entries

pro expire state source virtual destination

TCP 00:57 SYN_RECV 10.10.3.100:51167 10.10.3.135:80 192.168.3.137:80

TCP 00:57 SYN_RECV 10.10.3.100:51154 10.10.3.135:80 192.168.3.136:80

TCP 00:57 SYN_RECV 10.10.3.100:51158 10.10.3.135:80 192.168.3.137:80

TCP 00:57 SYN_RECV 10.10.3.100:51168 10.10.3.135:80 192.168.3.136:80

#发起访问时Real Server状态

[[email protected] ~]# ss -tan

State Recv-Q Send-Q Local Address:Port Peer Address:Port

LISTEN 0 128 *:80 *:*

SYN-RECV 0 0 10.10.3.135:80 10.10.3.100:51154

SYN-RECV 0 0 10.10.3.135:80 10.10.3.100:63149

SYN-RECV 0 0 10.10.3.135:80 10.10.3.100:51168

LISTEN 0 128 :::22 :::*

LISTEN 0 128 *:22 *:*

LISTEN 0 100 ::1:25 :::*

LISTEN 0 100 127.0.0.1:25 *:*

ESTAB 0 0 192.168.3.136:22 192.168.3.100:50658

以上是关于lvs-dr模式部署遇到问题，求教老鸟的主要内容，如果未能解决你的问题，请参考以下文章

LVS负载均衡群集部署之——DR模式的介绍及搭建步骤

直接进去正题！LVS-DR模式keppalived及LVS-DR-keepalived部署！

LVS负载均衡群集DR模式+Keepalived部署！

负载均衡集群之——LVS-DR的部署

负载均衡LVS-DR模式群集部署实操