Buildbot in Docker
前言
最近使用Buildbot做了一个自动测试的框架,为了部署方便,可以把测试框架做成Docker镜像,方便部署。这里记录下过程,供大家参考。
项目介绍
项目是一个Online的基于http的交易系统,其实就是一个Ad Exchange。然后为了对它进行自动化测试,我们会另外启动一个mock server模拟DSP,然后再启动一个mock client来向exchange发送请求,exchange会与后端DSP通讯后得到最终结果并且返回给client,client通过对收到结果的校验来进行测试。
另外,我们希望当exchange,mock client,mock dsp以及测试数据中的任意一个更新时,都去跑一遍完整的测试流程。
关键点
- 如何启动server
buildbot里直接启动web server会导致buildbot持续等待进程结束,无法到下一步。而这里我们需要启动exchange和mock dsp两个server。处理办法就是把这两个server利用supervisor来做成service,这样buildbot只需要作为一个client发出start/end命令即可。具体实施时,需要在docker镜像生成时去写supervisor的conf文件,添加service。而在buildbot进行build时,去调用supervisorctl start/end命令启停service。
Dockerfile以及master.cfg
- Dockerfile
from ubuntu:14.04
run /bin/echo -e "
deb http://mirrors.163.com/ubuntu/ trusty main multiverse restricted universe
deb http://mirrors.163.com/ubuntu/ trusty-security main multiverse restricted universe
deb http://mirrors.163.com/ubuntu/ trusty-updates main multiverse restricted universe
deb http://mirrors.163.com/ubuntu/ trusty-proposed universe restricted multiverse main
deb http://mirrors.163.com/ubuntu/ trusty-backports main multiverse restricted universe" >
/etc/apt/sources.list
run apt-get update
run DEBIAN_FRONTEND=noninteractive apt-get install -y python-pip python-dev
git sudo ssh nodejs npm supervisor lsof python3-setuptools python3-dev python3-pip
run pip install buildbot buildbot_slave
首先加入国内源,以及安装必须软件。我们的测试系统是基于python的,所以加入了python3的一些依赖。
run ln -s /usr/bin/nodejs /usr/bin/node
run mkdir /programs
workdir /programs
#install mock dsp && mock client
run git clone https://***.git
run git clone https://***.git
run cd mockdsp; python3 setup.py install
run cd mockclient; python3 setup.py install
这里mock dsp和client都会自动安装到系统path中,方便以后调用。
run mkdir -p /data/buildbot
workdir /data/buildbot
#create master
run buildbot create-master master
ADD master.cfg master/master.cfg
#create slaves
run buildslave create-slave exchange localhost:9989 exchange exchange
安装buildbot,并且添加自己写的master.cfg。
#write supervisor configuration file
run /bin/echo -e "
[program:exchange]
command=npm start
directory=/data/buildbot/exchange/builder-exchange/exchange
autostart=false
[program:mockDsp]
command=/usr/local/bin/mockDsp
directory=/tmp
[program:master]
command=twistd --nodaemon --no_save -y buildbot.tac
directory=/data/buildbot/master
[program:slave]
command=twistd --nodaemon --no_save -y buildbot.tac
directory=/data/buildbot/exchange
" >
/etc/supervisor/conf.d/buildbot.conf
把exchange, mockDSP都做成service,另外buildbot master和buildbot slave是肯定要做成service的,这样可以在一个docker中运行他们。
cmd ["/usr/bin/supervisord", "-n"]
启动supervisor。
- master.cfg
基本的配置比如slaves,端口,codebases我就不贴出来了。这里列出就列出supervisor启动用的Steps
####### BUILD STEPS
factory = util.BuildFactory()
...
factory.addStep(steps.ShellCommand(
command=['supervisorctl', 'start', 'exchange']
, description="starting exchange"
, descriptionDone="start exchange"
, workdir="exchange"))
factory.addStep(steps.ShellCommand(
command=["supervisorctl","restart","mock"]
, description="restarting rmock"
, descriptionDone="rmock restarted"
, workdir="rmock"))
#跑完测试后
factory.addStep(steps.ShellCommand(
command=["supervisorctl", "stop", "exchange"]
, descriptionDone="shutdown exchange"
, workdir="testData"))
#mock server会一直运行
然后把image build出来运行就行啦,就这么简单!