A highly available WEB cluster based on HAProxy + KeepAlived
I. Overview
1. Keepalived
Keepalived is a highly available solution for LVS services based on the VRRP protocol, which can be used to avoid single points of failure. An LVS service will have two servers running Keepalived, one is the master server (MASTER) and the other is the backup server (BACKUP), but it appears as a virtual IP to the outside, and the master server will send a specific message to the backup server. When the server does not receive this message, that is, when the primary server is down, the backup server will take over the virtual IP and continue to provide services, thus ensuring high availability. Keepalived is the perfect realization of VRRP.
2. Haproxy
HAProxy is a high-performance proxy server, which can provide 7-layer and 4-layer proxy, with multiple features such as healthcheck, load balancing, and excellent performance. It is used by many well-known Internet companies including Twitter, Reddit, StackOverflow, and GitHub.
KeepAlived is a high-availability solution that achieves high availability through VIP (ie virtual IP) and heartbeat detection. The principle is that there is a set of (two) servers, which are assigned the two roles of Master and Backup respectively. By default, the Master will bind VIP to its own network card to provide external services. Master and Backup will send heartbeat packets to each other at a certain time interval to detect the status of the other party. This time interval is generally 2 seconds. If Backup finds that the Master is down, then Backup will send an ARP packet to the gateway to bind the VIP To its own network card, Backup provides services to the outside world to achieve automatic failover. When the Master recovers, it will take over the service again.
2. Experimental environment
-
Experimental environment: prepare 4 servers, 2 haproxy+keepalived servers, respectively as the main and backup. One dynamic website server, one static website server.
-
The purpose of the experiment: realize that external users can access the web server through vip, and can realize the separation of dynamic and static according to the different access content. If the front-end scheduler is down, it can be quickly switched to let the server go online to take over all affairs without affecting user access!
| server | IP address | system version | Software version | |
| Front-end master scheduler |
eth0:172.16.8.5 vip:172.16.8.1 |
Centos6.5 |
keepalived-1.2.7-3.el6.x86_64 haproxy-1.4.24-2.el6.x86_64 |
|
| Front-end standby scheduler |
eth0:172.16.8.8 vip:172.16.8.1 |
Centos6.5 |
keepalived-1.2.7-3.el6.x86_64 haproxy-1.4.24-2.el6.x86_64 |
|
| Static server | eth0:172.16.8.7 | Centos6.5 | httpd | |
| Dynamic server | eth0:172.16.8.9 | Centos6.5 | httpd php | |
-
Turn off the firewall of each server
//Close iptables and SELINUX # service iptables stop # setenforce 0 # vim/etc/sysconfig/selinux --------------- SELINUX=disabled
3. haproxy and keepalived installation configuration
1. Time synchronization:
[root@node1 ~]# ntpdate 172.16.0.1//172.16.0.1 is the time server [root@node2 ~]# ntpdate 172.16.0.1
2. Install keepalived and haproxy on the 2 active and standby servers
[root@node1 ~]# yum -y install haproxy keepalived [root@node2 ~]# yum -y install haproxy keepalived
3. Keepalived installation and configuration
3.1. Configure on the main dispatch server:
-
Modify the keepalived configuration file
[root@node1 ~]# vim/etc/keepalived/keepalived.conf
//amend as below:
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost//Configure the administrator mailbox
}
notification_email_from kaadmin@localhost//Configure the sender
smtp_server 127.0.0.1//Configure mail server
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_mantaince_down {
script "[[ -e/etc/keepalived/down ]] && exit 1 || exit 0"
interval 1
weight -150//This script means that if there is a down file in the/etc/keepalived/directory, the priority of this master server will be lowered by 150 (you can set it by yourself), and let other slave servers become master servers
}
vrrp_instance VI_1 {
state MASTER//Configuration mode master is the master server
interface eth0
virtual_router_id 51//Virtual router id number
priority 220//Priority, the higher the priority of each node, the more likely it is to become the main server
advert_int 1
authentication {
auth_type PASS//Authentication method
auth_pass 1111
}
virtual_ipaddress {
172.16.8.1//Configure virtual ip, vip
}
track_script {
chk_mantaince_down//Check the script
}
//When the main server status changes, send the change information to the mail server, which can be viewed through the mail command
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
-
Create notify.sh script
[root@node1 ~]# vim/etc/keepalived/notify.sh
//Add the following content:
#!/bin/bash
# Author: MageEdu <linuxedu@foxmail.com>
# description: An example of notify script
#
vip=172.16.8.1
contact='root@localhost'
notify() {
mailsubject="`hostname` to be $1: $vip floating"
mailbody="`date'+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
case "$1" in
master)
notify master
exit 0
;;
backup)
notify backup
exit 0
;;
fault)
notify fault
exit 0
;;
*)
echo'Usage: `basename $0` {master|backup|fault}'
exit 1
;;
esac
Send this script to the slave server:
[root@node1 ~]# scp/etc/keepalived/notify.sh root@172.16.8.8:/etc/keepalived/
3.2, configure on the slave server
-
Modify the keepalived configuration file
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from kaadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_mantaince_down {
script "[[ -f/etc/keepalived/down ]] && exit 1 || exit 0"
interval 1
weight -150
}
vrrp_instance VI_1 {
state BACKUP//Modify to BACKUP
interface eth0
virtual_router_id 51
priority 160//Priority is lower than the main one
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
172.16.8.1
}
track_script {
chk_mantaince_down
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
3.3, test
-
Start the main and standby keepalived separately
[root@node1 ~]# service keepalived start [root@node2 ~]# service keepalived start
-
View ip
-
Create down file on the main server
[root@node1 ~]# touch/etc/keepalived/down [root@node1 ~]# ip a [root@node1 ~]# mail
3. Configure haprox
The haproxy configuration files of the master and slave servers must be the same
[root@node1 ~]# vim/etc/haproxy/haproxy.cfg
//amend as below:
#------------------------------------------------- --------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#------------------------------------------------- --------------------
#------------------------------------------------- --------------------
# Global settings
#------------------------------------------------- --------------------
global
# to have these messages end up in/var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the'-r' option to the SYSLOGD_OPTIONS in
#/etc/sysconfig/syslog
#
# 2) configure local2 events to go to the/var/log/haproxy.log
# file. A line like the following can be added to
#/etc/sysconfig/syslog
#
# local2.*/var/log/haproxy.log
#
//The above comment tells us how to configure the log
log 127.0.0.1 local2//Log output configuration, all logs are recorded in this machine, and output through local2
chroot/var/lib/haproxy//Change the current working directory, safe mode
pidfile/var/run/haproxy.pid//pid file
maxconn 4000//Maximum number of connections
user haproxy//User
group haproxy//group
daemon//Run haproxy in the background
# turn on stats unix socket
stats socket/var/lib/haproxy/stats
#------------------------------------------------- --------------------
# common defaults that all the'listen' and'backend' sections will
# use if not designated in their block
#------------------------------------------------- --------------------
defaults//Configure the default parameters, these parameters can be used to configure the frontend, backend, listen components
mode http//The default mode mode {tcp|http|health }, tcp is 4 layers, http is 7 layers, health will only return OK (note, health is obsolete)
log global//Use the globally defined log
option httplog//Log type http log format
option dontlognull//Do not record health check log information
option http-server-close//Actively close the http channel after each request
option forwardfor except 127.0.0.0/8//Do not record the log forwarded by this machine
option redispatch//After the server corresponding to serverId hangs up, it is forcibly directed to other healthy servers
retries 3//If the connection fails 3 times, the service is considered unavailable, or it can be set later
timeout http-request 10s//Request timeout
timeout queue 1m//Queue timeout
timeout connect 10s//Connection timeout
timeout client 1m//Client connection timeout
timeout server 1m//Server connection timeout
timeout http-keep-alive 10s//Long connection timeout
timeout check 10s//Check timeout
maxconn 3000//Maximum number of connections
listen stats//listen is a combination of Frontend and Backend. What is defined here is haproxy monitoring!
mode http//mode http
bind *:80//Bind monitoring ip and port
stats enable//Enable monitoring
stats hide-version//Hide the haproxy version
stats uri/haproxyadmin?stats//defined uri
stats realm Haproxy/Statistics//Prompt text on the password box of the statistics page
stats auth admin:admin//Authentication
stats admin if TRUE//Enable the management interface
frontend http//The front-end virtual node that receives the request. Frontend can directly specify the backend to use the backend according to the rules (can be dynamically selected). What is defined here is http service!
bind *:80//Bind monitoring ip and port
mode http//mode http
acl url_static path_beg -i/static/p_w_picpaths/javascript/stylesheets//acl is followed by the rule name to define access control
acl url_static path_end -i .jpg .gif .png .css .js .html
acl url_dynamic path_end -i .php
use_backend static if url_static//Satisfy the condition of url_static, then enable static backend
use_backend dynamic if url_dynamic meets the condition of url_dynamic, then dynamic backend is enabled
default_backend dynamic//The default backend defined
backend static//static scope
mode http
balance roundrobin
//banlance roundrobin polling, balance source saves the session value, supports static-rr, leastconn, first, uri and other parameters
option httpchk/index.html
//Detect the file, if it is not distributed to the background index.html, it will not be distributed to it
server web1 172.16.8.7:80 check inter 2000 rise 2 fall 3
backend dynamic
mode http
balance roundrobin
option httpchk/index.php
server web2 172.16.8.9:80 check inter 2000 rise 2 fall 3
3.1, modify the log file
//Modify the configuration file of the system log [root@haproxy ~]# vim/etc/sysconfig/rsyslog # Options for rsyslogd # Syslogd options are deprecated since rsyslog v3. # If you want to use them, switch to compatibility mode 2 by "-c 2" # See rsyslogd(8) for more details SYSLOGD_OPTIONS="-c 2 -r"
3.2, increase log equipment
[root@haproxy ~]# vim/etc/rsyslog.conf #Add a line local2.*/var/log/haproxy.log
3.3, restart the log service
[root@haproxy ~]# service rsyslog restart
3.4, check the configuration file syntax
[root@haproxy ~]# haproxy -c -f/etc/haproxy/haproxy.cfg Configuration file is valid
3.5. Send this configuration file to the slave server:
[root@node1 haproxy]# scp/etc/haproxy/haproxy.cfg root@172.16.8.8:/etc/haproxy/
4. Create haproxy+keepalived script:
This script realizes that when haproxy hangs up, haproxy can be started again, if it cannot be started again, keepalived will be completely closed and the VIP will
Slave processing.
[root@node1 ~]# vim/etc/keepalived/check_haproxy.sh //Add the following content --------------------- #!/bin/bash while: do hapid=`ps -C haproxy --no-header |wc -l` if [$hapid -eq 0 ]; then /usr/local/haproxy/sbin/haproxy -f/usr/local/haproxy/haproxy.cfg sleep 5 if [$hapid -eq 0 ]; then /etc/init.d/keepalived stop fi fi sleep 5 done -------------------- [root@node1 ~]# chmod 755/etc/keepalived/check_haproxy.sh [root@node1 ~]# nohup sh/etc/keepalived/check_haproxy.sh &
Send this script to the slave server
[root@node1 ~]# scp/etc/keepalived/check_haproxy.sh root@172.16.8.8:/etc/keepalived/
5. Configure dynamic and static servers
-
Static server (172.16.8.7)
[root@master html]# yum -y install httpd
Add a test page:
#vim/var/www/html/index.html //Add the following content <h1>www.web1.com</h1>
-
Dynamic server (172.16.8.9)
# yum -y install httpd php
Add a test page:
# vim/var/www/html/index.php //Add the following content: <h1>www.web2.com</h1> <?php phpinfo(); ?>
4. simulated fault test
//The master and slave servers start keepalived and haproxy [root@node1 ~]# service keepalived start [root@node1 ~]# service haproxy start [root@node2 ~]# service keepalived start [root@node2 ~]# service haproxy start //The backend server starts the web [root@master ~]# service httpd start [root@station142 ~]# service httpd start
-
Enter http://172.16.8.1/haproxyadmin?stats in the browser , the account password is admin
So we can manage the back-end server through this stats interface!
-
Dynamic and static separation test
When accessing a static page, jump to the web1 server (172.16.8.7)
Jump to web2 when accessing dynamic pages, (172.16.8.9)
-
Stop the haproxy service of the main server to see if it affects access!
[root@node1 keepalived]# service haproxy stop
The check_haproxy.sh script detects that the haproxy service is stopped, and stops the keepalived service, so that the VIP is transferred from the master server to the slave server, and user access is not affected!
At this point, the high-availability web cluster based on keepalived and haproxy has been completed. If the reader has enough hosts, there can be multiple dynamic and static page servers behind, and a cache server can be added to speed up user access!
Reprinted at: https://blog.51cto.com/584014981/1405309