Burrow 服务的安装部署
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
最近协助开发的同时帮忙把10个topic的数据使用5个topic的来工作。结果发现数据flume在手机数据时存在延迟消费的问题。因此遇到了不少坑，要是有个专门监控kafka consumer lag的开源软件就好了！当然我们通过kafka-consumer-groups.sh这个命令也可以查看到相应的参数，但每次都需要敲击命令行。无意间听网友说有LinkedIn公司开源有一款叫做Burrow都软件，专门是对kafka消费这延迟检查。
一.Burrow概述
1>.什么是Burrow
Burrow是Apache Kafka的监控伴侣，它将消费者滞后检查作为服务提供，无需指定阈值。它监控所有消费者的承诺抵消额，并根据需要计算这些消费者的状态。提供HTTP端点以按需请求状态，以及提供其他Kafka群集信息。还有可配置的通知程序可以通过电子邮件或HTTP调用将状态发送到另一个服务。
2>.Burrow特征
没有门槛！通过滑动窗口评估组。
多个Kafka群集支持
使用Kafka提交的偏移自动监视所有使用者
可配置支持Zookeeper提交的偏移量
可配置支持Storm-committed offsets
消费者组状态的HTTP端点，以及代理和消费者信息
可配置的电子邮件程序，用于发送特定组的警报
可配置的HTTP客户端，用于向所有组的另一个系统发送警报
3>.先决条件
Burrow是用Go编写的，所以在开始之前，你应该安装并设置Go。
二.安装Golang环境
1>.下载 go 的安装包
[root@node101.yinzhengjie.org.cn ~/Downloads]# wget https://dl.google.com/go/go1.12.1.linux-amd64.tar.gz --2019-03-28 16:09:35-- https://dl.google.com/go/go1.12.1.linux-amd64.tar.gz Resolving dl.google.com (dl.google.com)... 203.208.41.70, 203.208.41.72, 203.208.41.65, ... Connecting to dl.google.com (dl.google.com)|203.208.41.70|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 127906702 (122M) [application/octet-stream] Saving to: ‘go1.12.1.linux-amd64.tar.gz’ 100%[=========================================================================================================================================================================>] 127,906,702 2.27MB/s in 54s 2019-03-28 16:10:29 (2.28 MB/s) - ‘go1.12.1.linux-amd64.tar.gz’ saved [127906702/127906702] [root@node101.yinzhengjie.org.cn ~/Downloads]#
2>.解压go的安装包到指定目录
[root@node101.yinzhengjie.org.cn ~/Downloads]# tar -zxf go1.12.1.linux-amd64.tar.gz -C /yinzhengjie/softwares/ [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# ll /yinzhengjie/softwares/go/ total 208 drwxr-xr-x. 2 root root 4096 Mar 15 03:43 api -rw-r--r--. 1 root root 55358 Mar 15 03:43 AUTHORS drwxr-xr-x. 2 root root 4096 Mar 15 03:46 bin -rw-r--r--. 1 root root 1339 Mar 15 03:43 CONTRIBUTING.md -rw-r--r--. 1 root root 78132 Mar 15 03:43 CONTRIBUTORS drwxr-xr-x. 8 root root 4096 Mar 15 03:43 doc -rw-r--r--. 1 root root 5686 Mar 15 03:43 favicon.ico drwxr-xr-x. 3 root root 4096 Mar 15 03:43 lib -rw-r--r--. 1 root root 1479 Mar 15 03:43 LICENSE drwxr-xr-x. 13 root root 4096 Mar 15 03:43 misc -rw-r--r--. 1 root root 1303 Mar 15 03:43 PATENTS drwxr-xr-x. 6 root root 4096 Mar 15 03:46 pkg -rw-r--r--. 1 root root 1607 Mar 15 03:43 README.md -rw-r--r--. 1 root root 26 Mar 15 03:43 robots.txt drwxr-xr-x. 46 root root 4096 Mar 15 03:43 src drwxr-xr-x. 21 root root 12288 Mar 15 03:43 test -rw-r--r--. 1 root root 8 Mar 15 03:43 VERSION [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]#
3>.配置go到环境变量
[root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# mkdir /yinzhengjie/goproject [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# tail -5 /etc/profile #ADD GO_PATH export GO_HOME=/yinzhengjie/softwares/go export PATH=$PATH:$GO_HOME/bin export GOPATH=/yinzhengjie/goproject [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]#
[root@node101.yinzhengjie.org.cn ~/Downloads]# source /etc/profile [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]# go env GOARCH="amd64" GOBIN="" GOCACHE="/root/.cache/go-build" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOOS="linux" GOPATH="/yinzhengjie/goproject" GOPROXY="" GORACE="" GOROOT="/yinzhengjie/softwares/go" GOTMPDIR="" GOTOOLDIR="/yinzhengjie/softwares/go/pkg/tool/linux_amd64" GCCGO="gccgo" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build685793832=/tmp/go-build -gno-record-gcc-switches" [root@node101.yinzhengjie.org.cn ~/Downloads]# [root@node101.yinzhengjie.org.cn ~/Downloads]#
三.Burrow安装
1>.Burrow的github地址（https://github.com/linkedin/Burrow）
2>.安装git工具
[root@node101.yinzhengjie.org.cn ~]# yum -y install git Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com Resolving Dependencies --> Running transaction check ---> Package git.x86_64 0:1.8.3.1-20.el7 will be installed --> Processing Dependency: perl-Git = 1.8.3.1-20.el7 for package: git-1.8.3.1-20.el7.x86_64 --> Processing Dependency: perl(Term::ReadKey) for package: git-1.8.3.1-20.el7.x86_64 --> Processing Dependency: perl(Git) for package: git-1.8.3.1-20.el7.x86_64 --> Processing Dependency: perl(Error) for package: git-1.8.3.1-20.el7.x86_64 --> Running transaction check ---> Package perl-Error.noarch 1:0.17020-2.el7 will be installed ---> Package perl-Git.noarch 0:1.8.3.1-20.el7 will be installed ---> Package perl-TermReadKey.x86_64 0:2.30-20.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================================================== Installing: git x86_64 1.8.3.1-20.el7 updates 4.4 M Installing for dependencies: perl-Error noarch 1:0.17020-2.el7 base 32 k perl-Git noarch 1.8.3.1-20.el7 updates 55 k perl-TermReadKey x86_64 2.30-20.el7 base 31 k Transaction Summary =================================================================================================================================================================================================================== Install 1 Package (+3 Dependent packages) Total download size: 4.5 M Installed size: 22 M Downloading packages: (1/4): perl-Error-0.17020-2.el7.noarch.rpm | 32 kB 00:00:00 perl-TermReadKey-2.30-20.el7.x FAILED http://mirrors.cloud.aliyuncs.com/centos/7/os/x86_64/Packages/perl-TermReadKey-2.30-20.el7.x86_64.rpm: [Errno 14] curl#6 - "Could not resolve host: mirrors.cloud.aliyuncs.com; Unknown error" 0 B --:--:-- ETA Trying other mirror. (2/4): perl-TermReadKey-2.30-20.el7.x86_64.rpm | 31 kB 00:00:00 perl-Git-1.8.3.1-20.el7.noarch FAILED http://mirrors.cloud.aliyuncs.com/centos/7/updates/x86_64/Packages/perl-Git-1.8.3.1-20.el7.noarch.rpm: [Errno 14] curl#6 - "Could not resolve host: mirrors.cloud.aliyuncs.com; Unknown error" 0 B --:--:-- ETA Trying other mirror. (3/4): git-1.8.3.1-20.el7.x86_64.rpm | 4.4 MB 00:00:01 (4/4): perl-Git-1.8.3.1-20.el7.noarch.rpm | 55 kB 00:00:01 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 2.1 MB/s | 4.5 MB 00:00:02 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : 1:perl-Error-0.17020-2.el7.noarch 1/4 Installing : perl-TermReadKey-2.30-20.el7.x86_64 2/4 Installing : git-1.8.3.1-20.el7.x86_64 3/4 Installing : perl-Git-1.8.3.1-20.el7.noarch 4/4 Verifying : perl-Git-1.8.3.1-20.el7.noarch 1/4 Verifying : 1:perl-Error-0.17020-2.el7.noarch 2/4 Verifying : git-1.8.3.1-20.el7.x86_64 3/4 Verifying : perl-TermReadKey-2.30-20.el7.x86_64 4/4 Installed: git.x86_64 0:1.8.3.1-20.el7 Dependency Installed: perl-Error.noarch 1:0.17020-2.el7 perl-Git.noarch 0:1.8.3.1-20.el7 perl-TermReadKey.x86_64 0:2.30-20.el7 Complete! [root@node101.yinzhengjie.org.cn ~]#
2>.
3>.
4>.
5>.