Keepalived-LVS 能够提高集群的高可用性并增加后端检测功能、简化配置，满足常规需求。但Keepalived-LVS集群中，同一个VIP只能由一台设备进行宣告，为一主多备的架构，不能横向拓展集群的性能，为此我们引入OSPF来解决该问题。
OSPF(ECMP)
ECMP（Equal-CostMultipathRouting）等价多路径，存在多条不同链路到达同一目的地址的网络环境中，如果使用传统的路由技术，发往该目的地址的数据包只能利用其中的一条链路，其它链路处于备份状态或无效状态，并且在动态路由环境下相互的切换需要一定时间，而等值多路径路由协议可以在该网络环境下同时使用多条链路，不仅增加了传输带宽，并且可以无时延无丢包地备份失效链路的数据传输。
特点：
- 基于流的均衡负载
- 最大链路数受设备限制（最高16）
- 所有链路都active，故障链路自动剔除
LVS+OSPF(ECMP)
利用ECMP以上特性，可以将LVS集群进行横向拓展，利用quagga启ospf
为模拟集群环境，我们准备了六台虚拟机分别为Client、LVS-1、LVS-2、RealServer1、RealServer2、Router，VIP设为192.168.0.100
- Router ：192.168.0.1 192.168.1.1
- Client ：192.168.1.2
- LVS-1 ：192.168.0.2
- LVS-2 ：192.168.0.3
- RealServer1 ：192.168.0.4
- RealServer2 ：192.168.0.5
Router 配置
LVS-1、LVS-2与Router需处于同一ospf域中，通过 IP 192.168.0.1 与LVS1/LVS2/RealServer1/RealServer2 通讯，IP 192.168.1.1 与 Client通讯
[root@router ~]# echo 1 > /proc/sys/net/ipv4/ip_forward// 开启转发[root@router ~]# ifconfigens33: flags=4163 mtu 1500 inet 192.168.0.1 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::20c:29ff:fe6e:d10e prefixlen 64 scopeid 0x20 ether 00:0c:29:6e:d1:0e txqueuelen 1000 (Ethernet) RX packets 63921 bytes 5978914 (5.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24354 bytes 2334494 (2.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0ens38: flags=4163 mtu 1500 inet 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::20c:29ff:fe6e:d122 prefixlen 64 scopeid 0x20 ether 00:0c:29:6e:d1:22 txqueuelen 1000 (Ethernet) RX packets 60501 bytes 5206254 (4.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 18358 bytes 1432690 (1.3 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 1119 bytes 88568 (86.4 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1119 bytes 88568 (86.4 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0// ens33：192.168.0.1 与LVS1/LVS2/RealServer1/RealServer同一子网 ， ens38：192.168.1.1 与Client同一子网[root@LVS1 ~]# yum install quagga.x86_64 -y[root@router ~]# vi /etc/quagga/zebra.confhostname Routerpassword testenable password testlog file /var/log/quagga/zebra.logservice password-encryptioninterface ens33interface ens38access-list 1 permit 127.0.0.1ip prefix-list ANY seq 5 permit 0.0.0.0/0 le 32route-map ANY deny 10 match ip address prefix-list ANYip protocol ospf route-map ANYline vty access-class 1[root@router ~]# vi /etc/quagga/ospfd.confhostname Routerpassword testlog file /var/log/quagga/ospfd.loglog stdoutlog syslogservice password-encryptioninterface ens33 ip ospf hello-interval 1 ip ospf dead-interval 4 ip ospf priority 1 ip ospf cost 1router ospf ospf router-id 192.168.0.7 log-adjacency-changes network 192.168.0.0/24 area 0.0.0.0 network 192.168.1.0/24 area 0.0.0.0access-list 1 permit 127.0.0.1line vty access-class 1 RealServer 配置
realserver.sh 参考上一章节
[root@RealServer1 ~]# ./realserver.sh start[root@RealServer1 ~]# ifconfigens33: flags=4163 mtu 1500 inet 192.168.0.4 netmask 255.255.255.0 broadcast 192.168.0.255 inet6 fe80::20c:29ff:febd:38da prefixlen 64 scopeid 0x20 ether 00:0c:29:bd:38:da txqueuelen 1000 (Ethernet) RX packets 48635 bytes 4087456 (3.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 46101 bytes 5700308 (5.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10 loop txqueuelen 1000 (Local Loopback) RX packets 30 bytes 2613 (2.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 30 bytes 2613 (2.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo:0: flags=73 mtu 65536 inet 192.168.0.100 netmask 255.255.255.255 loop txqueuelen 1000 (Local Loopback) LVS 配置
由于我们使用ospf来实现高可用，不开启keepalived的vrrp功能（LVS2也同样配置），只是用其后端检测功能。
在keepalived-LVS集群的抢占模式下，Master在网卡上挂VIP并进行ARP广播，此时VIP对应的设备是唯一的。
但在OSPF-LVS集群中，Router根据ospf信息通过修改报文的目的mac地址转发到对应的LVS来实现均衡负载，并不根据VIP对应的ARP信息，所以对应的每台LVS将VIP挂在在lo上。
[root@LVS1 ~]# vi /etc/keepalived/keepalived.confglobal_defs { router_id LVS1 #路由器标识 script_user root enable_script_security}virtual_server 192.168.0.100 80 { delay_loop 5 lb_algo wrr lb_kind DR persistence_timeout 60 persistence_granularity 255.255.255.255 protocol tcp inhibit_on_failure on ha_suspend sorry_server 127.0.0.1 80 real_server 192.168.0.4 80 { weight 10 HTTP_GET{ url{ path / status_code 200 } connect_port 80 connect_timeout 2 retry 1 delay_before_retry 1 } } real_server 192.168.0.5 80 { weight 10 HTTP_GET{ url{ path / status_code 200 } connect_port 80 connect_timeout 2 retry 1 delay_before_retry 1 } }}[root@LVS1 ~]# systemctl reload keepalived[root@LVS1 ~]# ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.0.100:80 wrr persistent 60 -> 192.168.0.4:80 Route 10 0 0 -> 192.168.0.5:80 Route 10 0 0[root@LVS1 ~]# ifconfig lo:0 192.168.0.100 netmask 255.255.255.255 up//添加VIP到lo[root@LVS1 ~]# ip a1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet 192.168.0.100/32 scope global lo:0 valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: ens33: mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:af:6b:f7 brd ff:ff:ff:ff:ff:ff inet 192.168.0.2/24 brd 192.168.0.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:feaf:6bf7/64 scope link valid_lft forever preferred_lft forever[root@LVS1 ~]# route -nKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface0.0.0.0 192.168.0.1 0.0.0.0 UG 100 0 0 ens33192.168.0.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33 安装quagga并进行配置
[root@LVS1 ~]# yum install quagga.x86_64 -y[root@LVS1 ~]# vi /etc/quagga/zebra.confhostname LVS1# HOSTNAME改为IP也可以password testenable password testlog file /var/log/quagga/zebra.log#log syslogservice password-encryptioninterface ens33access-list 1 permit 127.0.0.1ip prefix-list ANY seq 5 permit 0.0.0.0/0 le 32route-map ANY deny 10 match ip address prefix-list ANYip protocol ospf route-map ANYline vty access-class 1[root@LVS1 ~]# vi /etc/quagga/ospfd.confhostname LVS1password testlog file /var/log/quagga/ospfd.loglog stdoutlog syslogservice password-encryptioninterface ens33 ip ospf hello-interval 1 ip ospf dead-interval 4 ip ospf priority 0 ip ospf cost 1router ospf ospf router-id 192.168.0.2 log-adjacency-changes network 192.168.0.2/24 area 0.0.0.0access-list 1 permit 127.0.0.1line vty access-class 1[root@LVS1 ~]# systemctl start zebra[root@LVS1 ~]# systemctl start ospfd同理我们配置完LVS2后进行检测ospf状态
[root@LVS1 ~]# vtyshHello, this is Quagga (version 0.99.22.4).Copyright 1996-2005 Kunihiro Ishiguro, et al.LVS1# show ip ospf neighbor Neighbor ID Pri State Dead Time Address Interface RXmtL RqstL DBsmL192.168.0.7 1 Full/DR 3.817s 192.168.0.1 ens33:192.168.0.2 0 0 0192.168.0.3 0 2-Way/DROther 3.518s 192.168.0.3 ens33:192.168.0.2 0 0 0LVS1# show ip ospf route============ OSPF network routing table ============N 192.168.0.0/24 [1] area: 0.0.0.0 directly attached to ens33N 192.168.0.100/32 [1] area: 0.0.0.0 directly attached to loN 192.168.1.0/24 [2] area: 0.0.0.0 via 192.168.0.1, ens33============ OSPF router routing table ========================= OSPF external routing table ===========在Route上查看路由
router# show ip routeCodes: K - kernel route, C - connected, S - static, R - RIP, O - OSPF, I - IS-IS, B - BGP, A - Babel, > - selected route, * - FIB routeK>* 0.0.0.0/0 via 192.168.0.10, ens33C>* 127.0.0.0/8 is directly connected, loO 192.168.0.0/24 [110/1] is directly connected, ens33, 00:11:04C>* 192.168.0.0/24 is directly connected, ens33O 192.168.0.100/32 [110/2] via 192.168.0.2, ens33 inactive, 00:00:11 via 192.168.0.3, ens33 inactive, 00:00:11O 192.168.1.0/24 [110/1] is directly connected, ens38, 00:12:19C>* 192.168.1.0/24 is directly connected, ens38//可以看到 到 192.168.0.100/32 的下一跳分配到 192.168.0.2、192.168.0.3Client 测试
[root@Client ~]# route -nKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface0.0.0.0 192.168.1.1 0.0.0.0 UG 100 0 0 ens33192.168.1.0 0.0.0.0 255.255.255.0 U 100 0 0 ens33[root@Client ~]# traceroute 192.168.0.100traceroute to 192.168.0.100 (192.168.0.100), 30 hops max, 60 byte packets 1 192.168.1.1 (192.168.1.1) 0.575 ms 0.258 ms 0.478 ms 2 192.168.0.100 (192.168.0.100) 1.901 ms 1.746 ms 1.370 ms[root@Client ~]# curl 192.168.0.100RealServer1 192.168.0.4自动化
以上是最基础的ospf-Lvs集群配置，在实际生产环境中会有自动化部署、监控、告警等需求，例如quagga的初始化配置
quagga 配置
HOSTNAME=`hostname`PASSWORD=shenyangchangkuan#获取默认路由对应网卡NIC=`/sbin/route -n|awk '$1=="default"||$1=="0.0.0.0"{print $NF}'|head -n 1`if [ -z $NIC ];then echo "get NIC err,NIC is null!"fi#获取该网卡IP#centos7改为下面这行#IP=`/sbin/ifconfig $NIC |grep "inet "|sed 's/:/ /'|awk '{print $2}'`IP=`/sbin/ifconfig $NIC|grep "inet addr"|sed 's/:/ /'|awk '{print $3}'`if [ -z $IP ];then echo "get IP err,IP is null!" exit 1fi#将该网卡及子网卡IP信息写入tmp/lvs_network.tmpecho "">/tmp/lvs_network.tmpfor cfg in ` ls -l /etc/sysconfig/network-scripts/ifcfg-${NIC}*|awk '{print $NF}'`do cat $cfg |grep IPADDR=|awk -F "=" '{print $2}' |sed 's/"//g'>>/tmp/lvs_network.tmpdone来源地址：https://blog.csdn.net/wxb880114/article/details/132478239
