跳转至

LVS HA(Keepalived)

2014-04-04 by dongnan

开始之前

Keepalived 概念及工作原理，PDF文档下载。

环境

LVS安装部署请参考这里。

功能

Keepalived 是运行在LVS之上，它的主要功能是实现RealServer(真实服务器)的故障隔离及Director(负载均衡器)间的FailOver(失败切换)。

  • keepalived是lvs的扩展项目，因此它们之间具备良好的兼容性。
  • 对RealServer健康检查，实现对失效机器/服务的故障隔离。
  • 负载均衡器之间的失败切换 FailOver。

部署

LVS-Master和LVS-Backup都需要安装 keepalived：

yum install keepalived

软件版本：

keepalived -v

Keepalived v1.2.7 (02/21,2013)

配置

备份配置文件

cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak

LVS-Master

编辑配置文件：

vim /etc/keepalived/keepalived.conf

配置内容类似如下：

awk '! /^(#|$)/' /etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
    router_id lvs-test
}

vrrp_sync_group VG1 {
    group {
        VI_1
    }
}

vrrp_instance VI_1 {
    state BACKUP
    nopreempt
    interface eth0
    track_interface {
        eth0
        eth1
    }
    virtual_router_id 45
    priority 150
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass dongnan
    }
    virtual_ipaddress {
        172.27.233.45/24 dev eth0 scope global
    }
}

virtual_server 172.27.233.45 80 {
    delay_loop 5
    lb_algo wrr
    lb_kind DR
    #persistence_timeout 60
    protocol TCP

    real_server 172.27.233.43 80 {
    weight 1
    TCP_CHECK  {
                   connect_timeout 3
                   nb_get_retry 3
                   delay_before_retry 3
                   connect_port 80
               }
     }

    real_server 172.27.233.44 80 {
    weight 1
    TCP_CHECK  {
                  connect_timeout 3
                  nb_get_retry 3
                  delay_before_retry 3
                  connect_port 80
               }
     }
}

LVS-Backup

LVS-Backup 与 LVS-Master 配置文件除了以下两项，其它配置项相同：

  • 去掉 nopreempt
  • 设置 priority 100

注意: nopreempt 只能用于 BACKUP 模式下，并且使用 nopreempt 的参数主机优先级也是最高的。

运行

启动 Keepalived

/etc/init.d/keepalived start

VIP

keepalived 帮我们绑定了 vip：

ip addr show eth0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether 00:50:56:9c:00:0c brd ff:ff:ff:ff:ff:ff
    inet 172.27.233.41/24 brd 172.27.233.255 scope global eth0
   inet 172.27.233.45/24 scope global secondary eth0
    inet6 fe80::250:56ff:fe9c:c/64 scope link
       valid_lft forever preferred_lft forever

LVS

keepalived 帮我们添加了lvs 规则：

ipvsadm -Ln

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.27.233.45:80 wrr
  -> 172.27.233.43:80             Route   1      0          0        
  -> 172.27.233.44:80             Route   1      0          0

RealServer

RealServer 脚本

脚本功能，开机启动自动完成绑定vip脚本。

FailOver测试

LVS

重启 LVS-Master，LVS-Backup自动切换成MASTER角色，VIP、LVS规则自动添加， 由于LVS-Master 设置不抢占，LVS-Master 启动后为Backup角色。

LVS日志

tail -f /var/log/messages

Mar 26 18:24:16 pn2 Keepalived_vrrp[2031]: VRRP_Instance(VI_1) Transition to MASTER STATE
Mar 26 18:24:16 pn2 Keepalived_vrrp[2031]: VRRP_Group(VG1) Syncing instances to MASTER state
Mar 26 18:24:17 pn2 Keepalived_vrrp[2031]: VRRP_Instance(VI_1) Entering MASTER STATE
Mar 26 18:24:17 pn2 Keepalived_vrrp[2031]: VRRP_Instance(VI_1) setting protocol VIPs.
Mar 26 18:24:17 pn2 Keepalived_vrrp[2031]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 172.27.233.45
Mar 26 18:24:17 pn2 Keepalived_healthcheckers[2030]: Netlink reflector reports IP 172.27.233.45 added
Mar 26 18:24:22 pn2 Keepalived_vrrp[2031]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 172.27.233.45

VIP地址

ip addr show eth0

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:50:56:9c:00:0d brd ff:ff:ff:ff:ff:ff
    inet 172.27.233.42/24 brd 172.27.233.255 scope global eth0
   inet 172.27.233.45/24 scope global secondary eth0
    inet6 fe80::250:56ff:fe9c:d/64 scope link
       valid_lft forever preferred_lft forever

LVS规则

ipvsadm -Ln --stats

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  172.27.233.45:80                   16       82        0     6996        0
  -> 172.27.233.43:80                    8       42        0     3532        0
  -> 172.27.233.44:80                    8       40        0     3464        0
RealServer

重启 RealServer，当健康检查发现http服务失效后被 keepalived 从 lvs 转发列表中删除，服务恢复后再被 keepalived 添加到 lvs 转发列表中。

LVS日志记录了 RealServer 故障后的日志：

tail /var/log/messages

Mar 26 18:19:11 pn2 Keepalived_healthcheckers[3035]: TCP connection to [172.27.233.43]:80 failed !!!
Mar 26 18:19:11 pn2 Keepalived_healthcheckers[3035]: Removing service [172.27.233.43]:80 from VS [172.27.233.45]:80
#
Mar 26 18:19:27 pn2 Keepalived_healthcheckers[3035]: TCP connection to [172.27.233.43]:80 success.
Mar 26 18:19:27 pn2 Keepalived_healthcheckers[3035]: Adding service [172.27.233.43]:80 to VS [172.27.233.45]:80
回到页面顶部