[Ansible实战]-ansible部署rsync全网备份项目

[Ansible实战]-ansible部署rsync全网备份项目

转自

第一章 项目需求

1.1 概述：

每天晚上12点整备份全网所有机器的系统配置文件和web服务器上的程序文件至备份服务器上

1.2 具体要求：

1)所有服务器的备份目录必须都为/backup。
2)要备份的系统配置文件包括但不限于：
a.定时任务服务的配置文件(/var/spool/cron/root)（适合web和nfs服务器）。
b.开机自启动的配置文件(/etc/rc.local)（适合web和nfs服务器）。
c.日常脚本的目录 (/server/scripts)（适合web和nfs服务器）。
d.防火墙iptables的配置文件(/etc/sysconfig/iptables)（适合web和nfs服务器）
e.其它一些需要备份的目录
3)Web服务器站点目录假定为(/var/html/www)。
4)Web服务器A访问日志路径假定为（/app/logs）
5)Web服务器保留打包后的7天的备份数据即可(本地留存不能多于7天，因为太多硬盘会满)
6)备份服务器上,保留每周一的所有数据副本，其它要保留6个月的数据副本。
7)备份服务器上要按照备份数据服务器的内网IP为目录保存备份，备份的文件按照时间名字保存。
8)需要确保备份的数据尽量完整正确，在备份服务器上对备份的数据进行检查，把备份的成功及失败结果信息发给系统管理员邮箱中

第二章 环境准备

管理服务器   IP:192.168.81.161 主机名：m01    centos7
存储服务器   IP:192.168.81.162 主机名：nfs01  centos7
WEB服务器   IP:192.168.81.163 主机名：nfs01   centos7
备份服务器   IP:192.168.81.165 主机名：nfs01  centos7

第三章 实现思路

3.1 思路：

3.2 ansible管理主机目录规划

#前期准备
[root@m01:/etc/ansible]# tree
.
├── rsync_client
│   ├── backup_sys.sh
│   ├── backup_web.sh
│   └── rsync.password
└── rsync_server
    ├── backup_server.sh
    ├── rsyncd.conf
    └── rsync.password
#角色目录规划
[root@m01:/etc/ansible/roles]# tree
.
├── rsync
│   ├── files
│   ├── handlers
│   ├── tasks
│   ├── templates
│   └── vars
├── rsync_client
│   ├── files
│   ├── handlers
│   ├── tasks
│   ├── templates
│   └── vars
└── rsync_server
    ├── files
    ├── handlers
    ├── tasks
    ├── templates
    └── vars

第四章 基础配置准备

4.1 rsync服务端准备工作

4.11 准备rsyncd.conf配置文件

[root@m01: ~]# vim /etc/ansible/rsync_server/rsyncd.conf
uid = rsync
gid = rsync
port = 873
fake super = yes
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 192.168.81.0/24
hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/rsync.password
 
[backup]
comment = "backup dir by michaleni"
path = /backup/

4.12 准备服务端密码认证文件

[root@m01:/etc/ansible/rsync_server]# echo "rsync_backup:123456" >rsync.password
[root@m01:/etc/ansible/rsync_server]# chmod 600 rsync.password

4.13 准备服务端脚本文件

[root@m01:/etc/ansible/rsync_server]# vim backup_server.sh
#!/bin/bash
#
#rsync_server script
#write by michaelni 2018-06-06

# del 180 day ago data
find /backup/ -type f -mtime +180 ! -name "*week1.tar.gz"|xargs rm 2>/dev/null

# check backup data
find /backup/ -type f -name "finger.txt"|xargs md5sum -c >/tmp/check.txt

# send check mail
mail -s "check backup info for $(date +%F -d -"1day") data" 906288036@qq.com </tmp/check.txt
## 4.2 rsync客户端准备工作
## 4.21 准备客户端密码认证文件
```bash
[root@m01:/etc/ansible/rsync_client]# echo "123456" >rsync.password
[root@m01:/etc/ansible/rsync_client]# chmod 600 rsync.password

4.22 准备客户端脚本文件

1）用来备份系统文件的脚本

[root@m01: ~]# vim /etc/ansible/rsync_client/backup_sys.sh

#!/bin/bash
#
#backup_sys script
#write by michaelni 2018-06-06

Backup_dir="/backup"
IP_info=$(hostname -I)  
## hostname -I查看内网网址，此处也可以使用awk进行过滤，也可以写成IP_info=`hostname -I`

# creat backup dir
mkdir -p $Backup_dir/$IP_info

# tar backup data
cd /
tar zchf $Backup_dir/$IP_info/system_backup_$(date +%F_week%w -d -"1day").tar.gz ./var/spool/cron/root ./etc/rc.local ./server/scripts ./etc/sysconfig/iptables

#del 7 day ago data
find $Backup_dir -type f -name +7|xargs rm 2>/dev/null  
##有可能会出现不存在七天之前的文件，此时删除会报错，所以将报错重定向为空

#creat finger file
find $Backup_dir/ -type f -mtime -1 ! -name "finger*"|xargs md5sum >$Backup_dir/$IP_info/finger.txt
##查找一天内的文件排除指纹文件(不对指纹文件进行校验）生成MD5校验值并重定向到校验值文件中

#backup push data info
rsync -az $Backup_dir/ rsync_backup@192.168.81.165::backup --password-file=/etc/rsync.password

2）用来备份服务应用数据的脚本

[root@m01: ~]# vim /etc/ansible/rsync_client/backup_web.sh

#!/bin/bash
#
#bacup_web_data script
#write by michaelni 2018-06-06

Backup_dir="/backup"
IP_info=$(hostname -I)  

# creat backup dir
mkdir -p $Backup_dir/$IP_info

# tar backup data
cd /
tar zchf $Backup_dir/$IP_info/system_backup_$(date +%F_week%w -d -"1day").tar.gz ./var/spool/cron/root ./etc/rc.local ./server/scripts ./etc/sysconfig/iptables
tar zchf $Backup_dir/$IP_info/www_backup_$(date +%F_week%w -d -"1day").tar.gz ./var/html/www
tar zchf /$Backup_dir/$IP_info/www_log_backup_$(date +%F_week%w -d -"1day").tar.gz ./app/logs
#del 7 day ago data
find $Backup_dir -type f -name +7|xargs rm 2>/dev/null 

#creat finger file
find $Backup_dir/ -type f -mtime -1 ! -name "finger*"|xargs md5sum >$Backup_dir/$IP_info/finger.txt

#backup push data info
rsync -az $Backup_dir/ rsync_backup@192.168.81.165::backup --password-file=/etc/rsync.password

4.3 编写主机清单

[rsync_server]
192.168.81.165
 
[rsync_client]
192.168.81.162
192.168.81.163
 
[rsync:children]
rsync_server
rsync_client

第五章 编写剧本

5.1 创建rsync角色标准目录

[root@m01:~]# cd /etc/ansible/roles/
[root@m01:/etc/ansible/roles]# mkdir -p {rsync,rsync_server,rsync_client}/{tasks,vars,files,handlers,templates}

5.2 编写main.yml

5.21 rsync公共部分

[root@m01:/etc/ansible/roles]# vim rsync/tasks/main.yml
- name: 01-install rsync
  yum: name=rsync state=installed

5.22 rsync_server部分

1)编写tasks目录下的main.yml

[root@m01:/etc/ansible/roles]# vim rsync_server/tasks/main.yml
- name: 01-push conf file & password file & bash file
  template: src={{ item.src }} dest={{ item.dest }} mode={{ item.mode }}
  loop:
    - { src: 'rsyncd.conf', dest: '/etc/', mode: '644' }
    - { src: '{{ passfile }}', dest: '/etc/', mode: '600' }
    - { src: '{{ scripts }}', dest: '/server/scripts/', mode: '755' }
  notify: restart rsyncd
# ignore_errors: yes
- name: 02-create user
  user: name=rsync create_home=no shell=/sbin/nologin
- name: 03-create rsync dir
  file: path={{ backupdir }} state=directory owner=rsync group=rsync
- name: 04-start rsync server
  service: name=rsyncd state=restarted enabled=yes
- name: 05-check server port
  shell: netstat -lntup | grep {{ Port_info }}
  register: get_server_port
- name: 06-display port info
  debug: msg={{ get_server_port.stdout_lines }}
- name: 07-set crontab
  cron: minute=5 hour=0 name='rsync backup' job='/bin/bash /server/scripts/backup_server.sh &>/dev/null'

2）编写vars目录下的main.yml

[root@m01: roles]# vim rsync_server/vars/main.yml
backupdir: /backup
Port_info: 873
passfile: rsync.password
scripts: backup_server.sh

3）准备files目录下的相关文件（包括配置文件，密码认证，脚本文件等）

[root@m01 roles]# cp -a /etc/ansible/rsync_server/* rsync_server/files/
[root@m01 roles]# ll ./rsync_server/files/
total 12
-rwxr-xr-x 1 root root 463 Apr  4 00:22 backup_server.sh
-rw-r--r-- 1 root root 530 Apr  3 23:35 rsyncd.conf
-rw------- 1 root root  23 Apr  4 00:17 rsync.password

4）编写templates目录下的相关文件

[root@m01 roles]# cp -a /etc/ansible/roles/rsync_server/files/* ./rsync_server/templates/
[root@m01 roles]# vim ./rsync_server/templates/rsyncd.conf
uid = rsync
gid = rsync
port = {{ Port_info }}
fake super = yes
use chroot = no
max connections = 200
timeout = 300
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
ignore errors
read only = false
list = false
hosts allow = 192.168.81.0/24
hosts deny = 0.0.0.0/32
auth users = rsync_backup
secrets file = /etc/{{ passfile }}
 
[backup]
comment = "backup dir by michaelni"
path = {{ backupdir }}
read_only = false

5）编写handlers目录下的main.yml

[root@m01：roles]# vim rsync_server/handlers/main.yml
- name: restart rsyncd
  service: name=rsyncd state=restarted
## 5.23 rsync_client部分
```bash
1）编写tasks目录下的main.yml

[root@m01: roles]# vim /etc/ansible/roles/rsync_client/tasks/main.yml
- name: 01-create scripts dir
  file: path=/server/scripts/ state=directory
- name: 02-push passfile & scriptfile
  template: src={{ item.src }} dest={{ item.dest }} mode={{ item.mode }}
  with_items:
    - { src: '{{ passfile }}', dest: '/etc/', mode: '600' }
    - { src: '{{ scripts }}', dest: '/server/scripts/', mode: '755' }
  #tags: t1
- name: 03-test rsync
  shell: rsync -az /etc/hosts rsync_backup@192.168.81.165::backup --port={{ Port_info }} --password-file=/etc/{{ passfile }}
- name: 04-set crontab
  cron: minute=0 hour=0 name='rsync backup' job='/bin/bash /server/scripts/{{ scripts }} &>/dev/null'

2）编写vars目录下的main.yml

[root@m01: roles]# vim /etc/ansible/roles/rsync_client/vars/main.yml
passfile: rsync.password
scripts: backup_sys.sh
#scripts: backup_web.sh
Port_info: 873

3）将客户端需要的文件准备好放到files目录下

[root@m01:/etc/ansible/roles]# cp -a ../rsync_client/* /etc/ansible/roles/rsync_client/files/

4)编写templates目录下文件

[root@m01:/etc/ansible/roles]# cp rsync_client/files/* rsync_client/templates/

[root@m01 templates]# vim backup_sys.sh
.......略........
rsync -az $Backup_dir/ rsync_backup@192.168.81.165::backup --port= {{ Port_info }} --password-file=/etc/ {{ passfile }}

[root@m01 templates]# vim backup_web.sh
.......略........
rsync -az $Backup_dir/ rsync_backup@192.168.81.165::backup --port= {{ Port_info }} --password-file=/etc/ {{ passfile }}

5.24 编写raync主剧本文件

[root@m01 roles]# vim /etc/ansible/roles/site_rsync.yml
- hosts: rsync
  roles:
    - rsync
- hosts: rsync_server
  roles:
    - rsync_server
- hosts: rsync_client
  roles:
    - rsync_client

5.25 测试
测试：将端口号改为999，看看配置文件和脚本里的端口是否更改，能否正常备份数据

[root@m01 roles]# vim rsync_server/vars/main.yml
[root@m01 roles]# vim rsync_client/vars/main.yml
passfile: rsync.password
scripts: backup.sh
Port_info: 888