文本处理三剑客之sed知识概述
三剑客sed命令概述介绍
三剑客sed命令执行原理命令基本语法
三剑客sed命令实操练习增删改查
一sed命令概述
sed - stream editor for filtering and transforming text
1）可以查找过滤筛选出指定的信息
2）擅长编辑文件内容信息
3）擅长对文件行进行操作
命令语法结构：sed [参数] ' 条件 指令' 文件信息
定义条件：根据行号 根据内容 根据正则信息
定义指令：满足条件的信息处理动作（添加删除修改输出）
二sed命令执行原理
sed命令执行过程核心原则：找什么信息，做什么事情
找什么信息==定义查找条件
做什么事情==操作执行命令（替换删除增加输出）
三sed命令的查询操作
实际上是输入操作
（一）根据行号筛选单行信息
[root@centos71 ~]# sed -n '3p' /etc/selinux/config
# SELINUX= can take one of these three values:
（二）根据行号筛选多行信息
连续的多行信息
[root@centos71 ~]# sed -n '2,4p' /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
不连续的多行信息
[root@centos71 ~]# cat -n /etc/selinux/config 1 2 # This file controls the state of SELinux on the system. 3 # SELINUX= can take one of these three values: 4 # enforcing - SELinux security policy is enforced. 5 # permissive - SELinux prints warnings instead of enforcing. 6 # disabled - No SELinux policy is loaded. 7 SELINUX=enforcing 8 # SELINUXTYPE= can take one of three values: 9 # targeted - Targeted processes are protected, 10 # minimum - Modification of targeted policy. Only selected processes are protected. 11 # mls - Multi Level Security protection. 12 SELINUXTYPE=targeted 13 14 15 [root@centos71 ~]# sed -n '1p;3p;5p' /etc/selinux/config # SELINUX= can take one of these three values: # permissive - SELinux prints warnings instead of enforcing.
（三）根据字符筛选单行信息
/ /这种结构和搜索替换是类似的，搜索替换包含了搜索
搜索替换sed 's///g'
[root@centos71 test]# sed -n '/dis/p' selinux.txt
# disabled - No SELinux policy is loaded.
SELINUX=disabled
（四）根据字符筛选多行信息
连续的多行信息
筛选从dis开头，target结尾的行
[root@centos71 test]# cat selinux.txt
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
[root@centos71 test]# sed -n '/dis/,/target/p' selinux.txt
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
[root@centos71 test]#
不连续的多行信息
[root@centos71 ~]# sed -n '/dis/p;/en/p' /etc/selinux/config
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
区间范围查找，搜索的起始字符一定要的，结束字符不一定要
有头无尾可以（壁虎），有尾无头不可以，直接死了
是贪婪匹配，每行进行过滤
四sed命令删除操作
（一）根据行号进行单行删除
[root@centos71 test]# cat -n selinux.txt
1
2 # This file controls the state of SELinux on the system.
3 # SELINUX= can take one of these three values:
4 # enforcing - SELinux security policy is enforced.
5 # permissive - SELinux prints warnings instead of enforcing.
6 # disabled - No SELinux policy is loaded.
7 SELINUX=disabled
8 # SELINUXTYPE= can take one of three values:
9 # targeted - Targeted processes are protected,
10 # minimum - Modification of targeted policy. Only selected processes are protected.
11 # mls - Multi Level Security protection.
12 SELINUXTYPE=targeted
13
14
15
[root@centos71 test]# sed '6d' selinux.txt
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
（二）根据行号进行多行删除
连续多行
注意操作是在内存的模式空间操作，对磁盘的block操作才真正操作
[root@centos71 test]# cat -n selinux.txt
1
2 # This file controls the state of SELinux on the system.
3 # SELINUX= can take one of these three values:
4 # enforcing - SELinux security policy is enforced.
5 # permissive - SELinux prints warnings instead of enforcing.
6 # disabled - No SELinux policy is loaded.
7 SELINUX=disabled
8 # SELINUXTYPE= can take one of three values:
9 # targeted - Targeted processes are protected,
10 # minimum - Modification of targeted policy. Only selected processes are protected.
11 # mls - Multi Level Security protection.
12 SELINUXTYPE=targeted
13
14
15
[root@centos71 test]# sed '1,11d' selinux.txt
SELINUXTYPE=targeted
不连续多行
[root@centos71 test]# cat -n selinux.txt
1
2 # This file controls the state of SELinux on the system.
3 # SELINUX= can take one of these three values:
4 # enforcing - SELinux security policy is enforced.
5 # permissive - SELinux prints warnings instead of enforcing.
6 # disabled - No SELinux policy is loaded.
7 SELINUX=disabled
8 # SELINUXTYPE= can take one of three values:
9 # targeted - Targeted processes are protected,
10 # minimum - Modification of targeted policy. Only selected processes are protected.
11 # mls - Multi Level Security protection.
12 SELINUXTYPE=targeted
13
14
15
[root@centos71 test]# sed '7d;12d' selinux.txt
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
五sed命令添加操作
（一）根据行号单行添加信息
巧记：a表示add+after
[root@centos71 test]# cat -n selinux.txt
1
2 # This file controls the state of SELinux on the system.
3 # SELINUX= can take one of these three values:
4 # enforcing - SELinux security policy is enforced.
5 # permissive - SELinux prints warnings instead of enforcing.
6 # disabled - No SELinux policy is loaded.
7 SELINUX=disabled
8 # SELINUXTYPE= can take one of three values:
9 # targeted - Targeted processes are protected,
10 # minimum - Modification of targeted policy. Only selected processes are protected.
11 # mls - Multi Level Security protection.
12 SELINUXTYPE=targeted
13
14
15
[root@centos71 test]# sed '1a hahahah' selinux.txt
hahahah
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
（二）根据行号多行添加信息
连续多行
[root@centos71 ~]# cat -n /etc/issue
1 S
2 Kernel
on an m
3
[root@centos71 ~]# sed '1,2aHAPPY' /etc/issue
S
HAPPY
Kernel
on an m
HAPPY
不连续多行
要加-e
[root@centos71 ~]# cat -n /etc/issue
1 S
2 Kernel
on an m
3
[root@centos71 ~]# sed -e '1aHAPPY' -e '2aHAPPY' -e '3aHAPPY' /etc/issue
S
HAPPY
Kernel
on an m
HAPPY
HAPPY
在指定行后面添加多行信息
[root@centos71 test]# sed '1a hahaha
xixixi' selinux.txt
hahaha
xixixi
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
在指定行前面插入信息
i插入，联系vim的i就是插入
[root@centos71 test]# sed '1i hahaha
xixixi' selinux.txt
hahaha
xixixi
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
企业常见配置文件添加信息方法
在文件的最后一行添加内容
$表示最后一行；在最后一行后面附加新的信息
[root@centos71 ~]# cat -n /etc/issue
1 S
2 Kernel
on an m
3
[root@centos71 ~]# sed '$ahahaxixi' /etc/issue
S
Kernel
on an m
hahaxixi
基础命令是最低级别的，不能识别正则符号
，会认为是字符
grep只能识别基础正则
sed只能识别基础正则
awk统计分析命令，是高级命令，可以识别基础正则和扩展正则
这种情况类似于高级语言和机器语言
六sed命令修改操作
（一）根据单行信息搜索替换
[root@centos71 test]# sed '7s#disabled#enforcing#g' selinux.txt
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
（二）利用正则将信息匹配出来再做替换
[root@centos71 ~]# cat -n /etc/selinux/config
1
2 # This file controls the state of SELinux on the system.
3 # SELINUX= can take one of these three values:
4 # enforcing - SELinux security policy is enforced.
5 # permissive - SELinux prints warnings instead of enforcing.
6 # disabled - No SELinux policy is loaded.
7 SELINUX=disabled
8 # SELINUXTYPE= can take one of three values:
9 # targeted - Targeted processes are protected,
10 # minimum - Modification of targeted policy. Only selected processes are protected.
11 # mls - Multi Level Security protection.
12 SELINUXTYPE=targeted
13
14
15
[root@centos71 ~]# sed '2,6s/#/abc/g' /etc/selinux/config
abc This file controls the state of SELinux on the system.
abc SELINUX= can take one of these three values:
abc enforcing - SELinux security policy is enforced.
abc permissive - SELinux prints warnings instead of enforcing.
abc disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
（三）利用后向引用前项做替换——修改网卡地址
[root@centos71 ~]# sed -r '/IPADDR/s#(.*=).*#1192.168.111.200#g' /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=eth0
UUID=9d9e2656-f3ac-4f75-8722-3136d239985d
DEVICE=eth0
ONBOOT=yes
IPADDR=192.168.111.200
PREFIX=24
GATEWAY=10.0.0.254
IPV6_PRIVACY=no
DNS1=223.5.5.5
练习——sed批量创建用户信息test01 .. test10并设置随机密码(6位)
（一）创建用户
[root@centos71 ~]# seq -w 10
01
02
03
04
05
06
07
08
09
10
[root@centos71 test]# seq -w 10 | sed -r 's#(.*)# useradd test1#g'
useradd test01
useradd test02
useradd test03
useradd test04
useradd test05
useradd test06
useradd test07
useradd test08
useradd test09
useradd test10
（二）设置随机密码
[root@centos71 ~]# # seq -w 10|sed -r 's#(.*)#useradd test1;echo password | passwd --stdin test1#g'
（三）保存随机密码
注意因为设置的是随机密码，所以要使用变量
[root@centos71 ~]# seq -w 10 | sed -r 's#(.*)#useradd test1;passwd=$(tr -cd 'a-zA-Z0-9' </dev/urandom|head -c 6);
echo $passwd | passwd --stdin test1; echo test1 $passwd >>/tmp/user_passwd#g'
练习——实现批量修改文件名称
将文件名称扩展名改为jpg
问题我的操作可以吗？
[root@centos71 test]# touch happy{01..10}.txt
[root@centos71 test]# ls happy*
happy01.txt happy03.txt happy05.txt happy07.txt happy09.txt happy.txt
happy02.txt happy04.txt happy06.txt happy08.txt happy10.txt
[root@centos71 test]# ls happy* | sed -r 's#(.*.)txt#1jpg#g'
happy01.jpg
happy02.jpg
happy03.jpg
happy04.jpg
happy05.jpg
happy06.jpg
happy07.jpg
happy08.jpg
happy09.jpg
happy10.jpg
happy.jpg
[root@centos71 test]# sed -r 's#(.*.)txt#1jpg#g' happy.txt
happy01.jpg
happy02.jpg
happy03.jpg
happy04.jpg
happy05.jpg
happy06.jpg
happy07.jpg
happy08.jpg
happy09.jpg
happy10.jpg
法二
[root@centos71 test]# ls happy*
happy01.txt happy03.txt happy05.txt happy07.txt happy09.txt happy.txt
happy02.txt happy04.txt happy06.txt happy08.txt happy10.txt
[root@centos71 test]# ls happy* | sed -r 's#(.*)txt#mv 1txt 1jpg#g'mv happy01.txt happy01.jpg
mv happy02.txt happy02.jpg
mv happy03.txt happy03.jpg
mv happy04.txt happy04.jpg
mv happy05.txt happy05.jpg
mv happy06.txt happy06.jpg
mv happy07.txt happy07.jpg
mv happy08.txt happy08.jpg
mv happy09.txt happy09.jpg
mv happy10.txt happy10.jpg
[root@centos71 test]# ls happy* | sed -r 's#(.*)txt#mv 1txt 1jpg#g' | bash
[root@centos71 test]# ls happy*
happy01.jpg happy03.jpg happy05.jpg happy07.jpg happy09.jpg happy.txt
happy02.jpg happy04.jpg happy06.jpg happy08.jpg happy10.jpg
练习——利用sed命令取出IP地址
（一）取出有IP地址所在行
[root@centos71 ~]# ip a s eth0|sed -n '3p'
inet 10.0.0.200/24 brd 10.0.0.255 scope global noprefixroute eth0
（二）取出IP地址
将前面/后面减掉，简言之掐头去尾留中间
[root@centos71 ~]# ip a s eth0 | sed -n '3p' | sed -r 's#.* (.*)/.*#1#g'
10.0.0.200
巧记：
双引号比单引号更多，作用更大,比如可以调用变量
七sed命令使用忠告
（一）sed操作时候要做备份
真正删除
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if SUFFIX supplied)
-i---将模式空间操作影响到磁盘中也就是将模式空间信息--覆盖--磁盘文件中
在编辑文件时，直接多文件进行备份sed -i.bak
注意sed命令在使用时用到多个参数，要将-i参数写在所有参数后面
下面对文件进行搜索替换的同时也进行了备份
[root@centos71 test]# cat -n /etc/selinux/config
1
2 # This file controls the state of SELinux on the system.
3 # SELINUX= can take one of these three values:
4 # enforcing - SELinux security policy is enforced.
5 # permissive - SELinux prints warnings instead of enforcing.
6 # disabled - No SELinux policy is loaded.
7 SELINUX=disabled
8 # SELINUXTYPE= can take one of three values:
9 # targeted - Targeted processes are protected,
10 # minimum - Modification of targeted policy. Only selected processes are protected.
11 # mls - Multi Level Security protection.
12 SELINUXTYPE=targeted
13
14
15
[root@centos71 test]# sed -ri.bak '7s#disabled#enforcing#' /etc/selinux/config
[root@centos71 test]# ls /etc/selinux/config*
/etc/selinux/config /etc/selinux/config.bak
[root@centos71 test]# ls /etc/selinux/config* -l
-rw-r--r-- 1 root root 543 Dec 19 19:43 /etc/selinux/config
-rw-r--r-- 1 root root 542 Dec 17 16:59 /etc/selinux/config.bak
[root@centos71 test]# diff /etc/selinux/config*
7c7
< SELINUX=enforcing
---
> SELINUX=disabled
（二）sed在使用参数时不要一起使用参数-i、-n
如果做了第1步，那么第2步做了还可以恢复，但是对企业还是会有损失
所以小心驶得万年船
八sed特别说明
注意单引号、双引号、三引号的作用是一样的，查看执行过程
[root@centos71 test]# cat sed.test.sh
#/bin/bash
echo single quotes
sed -n '/127.0.0.1/p' /etc/hosts
echo tow quotes
sed -n ''/127.0.0.1/p'' /etc/hosts
echo three quotes
sed -n '''/127.0.0.1/p''' /etc/hosts
[root@centos71 test]# bash -x sed.test.sh
+ echo single quotes
single quotes
+ sed -n /127.0.0.1/p /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
+ echo tow quotes
tow quotes
+ sed -n /127.0.0.1/p /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
+ echo three quotes
three quotes
+ sed -n /127.0.0.1/p /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
[root@centos71 test]# cat -n /etc/hosts
1 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
2 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6