• mesos框架编译部署


    mesos是什么呢?

    一个分布式调度框架,让你编写代码时面对整个集群像面对一台机器那么简单。所有的运行,资源调度都可以由它来帮你搞掂。

    1.mesos安装有两种方式:

    1)参考官网的getstart,下载源码然后进行编译

    http://mesos.apache.org/gettingstarted/

    这个就不多说,下载源码,configue、make、make install就好了。

    2)使用mesosphere编译好的包,直接安装

    https://mesosphere.com/downloads/  点mesos的get statred

    mesosphere造福人类,ubuntu安装如下:

    # Setup deb source
    sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
    DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
    CODENAME=$(lsb_release -cs)
    
    # Add the repository
    echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main" | 
      sudo tee /etc/apt/sources.list.d/mesosphere.list
    sudo apt-get -y update
    
    #install
    sudo apt-get -y install mesos

    so easy!

    2.部署运行

    1)单机运行模式

    #如果是一键安装
    #通过下面命令,就可以开启本地模式
    mesos-local 
    
    #代码安装则在build目录下面脚本启动
    ./bin/mesos-local.sh

    2) 集群运行模式

    #源码编译模式,进入build
    #必须加上 --ip ,否则都是127.0.0.1 ,这样在集群里面交换信息的时候,就会出现不能识别(都以为是本机的端口)
    # Start mesos master.
    $ ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos
    # Start mesos slave.
    $ ./bin/mesos-slave.sh --master=127.0.0.1:5050
    #命令行更多的参数参考官方文档 http://mesos.apache.org/documentation/latest/configuration/
    #可以使用自带的mesos-daemon.sh启动,但是运行信息全部都会写入null中去
    #例如:
    mesos-daemon.sh mesos-master --work_dir=/var/lib/mesos --ip=10.230.136.197
    mesos-daemon.sh mesos-slave --master=10.230.136.197:5050 --ip=10.29.23.28
    #也可以自己编写脚本,把运行信息写入到指定文件,便于查看
    nohup mesos-slave --master=10.230.136.197:5050 --ip=10.29.23.28 > mesos.info 2>&1

      $./bin/mesos-master.sh --work_dir=/var/lib/mesos --ip=10.230.136.197
      I1105 04:57:33.349346 7152 main.cpp:229] Build: 2015-09-19 00:50:33 by qingpingzhang
      I1105 04:57:33.349531 7152 main.cpp:231] Version: 0.24.0
      I1105 04:57:33.349786 7152 main.cpp:252] Using 'HierarchicalDRF' allocator
      I1105 04:57:33.359891 7152 leveldb.cpp:176] Opened db in 9.846621ms
      I1105 04:57:33.368098 7152 leveldb.cpp:183] Compacted db in 8.143958ms
      I1105 04:57:33.368237 7152 leveldb.cpp:198] Created db iterator in 66033ns

    I1105 04:57:33.368304  7152 leveldb.cpp:204] Seeked to beginning of db in 23717ns                                                                                                                                
    I1105 04:57:33.368751  7152 leveldb.cpp:273] Iterated through 3 keys in the db in 411999ns
    I1105 04:57:33.368988  7152 replica.cpp:744] Replica recovered with log positions 8001 -> 8002 with 0 holes and 0 unlearned
    I1105 04:57:33.371697  7173 recover.cpp:449] Starting replica recovery
    I1105 04:57:33.372709  7152 main.cpp:465] Starting Mesos master
    I1105 04:57:33.373004  7173 recover.cpp:475] Replica is in VOTING status
    I1105 04:57:33.373239  7173 recover.cpp:464] Recover process terminated
    I1105 04:57:33.377918  7169 master.cpp:378] Master 20151105-045733-3314083338-5050-7152 (ip-10-230-136-197.ec2.internal) started on 10.230.136.197:5050
    I1105 04:57:33.377959  7169 master.cpp:380] Flags at startup: --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate="false" --authenticate_slaves="false" --authenticators="crammd5" --authorizers="local" --framework_sorter="drf" --help="false" --initia
    lize_driver_logging="true" --ip="10.230.136.197" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_slave_ping_timeouts="5" --port="5050" --quiet="false" --recovery_slave_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1m
    ins" --registry_store_timeout="5secs" --registry_strict="false" --root_submissions="true" --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins" --user_sorter="drf" --version="false" --webui_dir="/home/qingpingzhang/dev/mesos-0.24.0/build/../src/webui" --work_d
    ir="/var/lib/mesos" --zk_session_timeout="10secs"
    I1105 04:57:33.378235  7169 master.cpp:427] Master allowing unauthenticated frameworks to register
    I1105 04:57:33.378258  7169 master.cpp:432] Master allowing unauthenticated slaves to register
    I1105 04:57:33.378309  7169 master.cpp:469] Using default 'crammd5' authenticator
    W1105 04:57:33.378353  7169 authenticator.cpp:505] No credentials provided, authentication requests will be refused.
    I1105 04:57:33.378376  7169 authenticator.cpp:512] Initializing server SASL
    I1105 04:57:33.390660  7172 master.cpp:1525] The newly elected leader is master@10.230.136.197:5050 with id 20151105-045733-3314083338-5050-7152
    I1105 04:57:33.390744  7172 master.cpp:1538] Elected as the leading master!
    I1105 04:57:33.390810  7172 master.cpp:1308] Recovering from registrar
    I1105 04:57:33.391114  7174 registrar.cpp:311] Recovering registrar
    I1105 04:57:33.392326  7173 log.cpp:661] Attempting to start the writer
    I1105 04:57:33.394917  7169 replica.cpp:477] Replica received implicit promise request with proposal 6
    I1105 04:57:33.397905  7169 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 2.948589ms
    I1105 04:57:33.397948  7169 replica.cpp:345] Persisted promised to 6
    I1105 04:57:33.398849  7174 coordinator.cpp:231] Coordinator attemping to fill missing position
    I1105 04:57:33.399190  7173 log.cpp:677] Writer started with ending position 8002
    I1105 04:57:33.401041  7170 leveldb.cpp:438] Reading position from leveldb took 76286ns
    I1105 04:57:33.401178  7170 leveldb.cpp:438] Reading position from leveldb took 28917ns
    I1105 04:57:33.403611  7172 registrar.cpp:344] Successfully fetched the registry (485B) in 12.319232ms
    I1105 04:57:33.404095  7172 registrar.cpp:443] Applied 1 operations in 136495ns; attempting to update the 'registry'
    I1105 04:57:33.406857  7168 log.cpp:685] Attempting to append 557 bytes to the log
    I1105 04:57:33.407032  7174 coordinator.cpp:341] Coordinator attempting to write APPEND action at position 8003
    I1105 04:57:33.408102  7167 replica.cpp:511] Replica received write request for position 8003
    I1105 04:57:33.410161  7167 leveldb.cpp:343] Persisting action (577 bytes) to leveldb took 1.985457ms
    I1105 04:57:33.410202  7167 replica.cpp:679] Persisted action at 8003
    I1105 04:57:33.410912  7170 replica.cpp:658] Replica received learned notice for position 8003
    I1105 04:57:33.412888  7170 leveldb.cpp:343] Persisting action (579 bytes) to leveldb took 1.931473ms
    I1105 04:57:33.412925  7170 replica.cpp:679] Persisted action at 8003
    I1105 04:57:33.412955  7170 replica.cpp:664] Replica learned APPEND action at position 8003
    I1105 04:57:33.414744  7167 registrar.cpp:488] Successfully updated the 'registry' in 10.496768ms
    I1105 04:57:33.415020  7167 registrar.cpp:374] Successfully recovered registrar
    I1105 04:57:33.415087  7171 log.cpp:704] Attempting to truncate the log to 8003
    I1105 04:57:33.415216  7172 coordinator.cpp:341] Coordinator attempting to write TRUNCATE action at position 8004
    I1105 04:57:33.415881  7168 replica.cpp:511] Replica received write request for position 8004
    I1105 04:57:33.416004  7171 master.cpp:1335] Recovered 2 slaves from the Registry (518B) ; allowing 10mins for slaves to re-register
    I1105 04:57:33.417834  7168 leveldb.cpp:343] Persisting action (18 bytes) to leveldb took 1.903245ms
    I1105 04:57:33.417870  7168 replica.cpp:679] Persisted action at 8004
    ./bin/mesos-master.sh --work_dir=/var/lib/mesos --ip=10.230.136.197
    I1105 04:57:33.349346  7152 main.cpp:229] Build: 2015-09-19 00:50:33 by qingpingzhang
    I1105 04:57:33.349531  7152 main.cpp:231] Version: 0.24.0
    I1105 04:57:33.349786  7152 main.cpp:252] Using 'HierarchicalDRF' allocator
    I1105 04:57:33.359891  7152 leveldb.cpp:176] Opened db in 9.846621ms
    I1105 04:57:33.368098  7152 leveldb.cpp:183] Compacted db in 8.143958ms
    I1105 04:57:33.368237  7152 leveldb.cpp:198] Created db iterator in 66033ns
    I1105 04:57:33.418318  7169 replica.cpp:658] Replica received learned notice for position 8004
    I1105 04:57:33.419096  7169 leveldb.cpp:343] Persisting action (20 bytes) to leveldb took 608618ns
    I1105 04:57:33.419170  7169 leveldb.cpp:401] Deleting ~2 keys from leveldb took 41579ns
    I1105 04:57:33.419198  7169 replica.cpp:679] Persisted action at 8004
    I1105 04:57:33.419229  7169 replica.cpp:664] Replica learned TRUNCATE action at position 8004
    I1105 04:57:41.857591  7173 master.cpp:3636] Registering slave at slave(1)@10.29.23.28:5051 (ip-10-29-23-28.ec2.internal) with id 20151105-045733-3314083338-5050-7152-S0
    I1105 04:57:41.858408  7171 registrar.cpp:443] Applied 1 operations in 111553ns; attempting to update the 'registry'
    I1105 04:57:41.860332  7167 log.cpp:685] Attempting to append 737 bytes to the log
    I1105 04:57:41.860429  7174 coordinator.cpp:341] Coordinator attempting to write APPEND action at position 8005
    I1105 04:57:41.861102  7171 replica.cpp:511] Replica received write request for position 8005
    I1105 04:57:41.861977  7171 leveldb.cpp:343] Persisting action (757 bytes) to leveldb took 772697ns
    I1105 04:57:41.862018  7171 replica.cpp:679] Persisted action at 8005
    I1105 04:57:41.862601  7168 replica.cpp:658] Replica received learned notice for position 8005
    I1105 04:57:41.864603  7168 leveldb.cpp:343] Persisting action (759 bytes) to leveldb took 1.946092ms
    I1105 04:57:41.864646  7168 replica.cpp:679] Persisted action at 8005
    I1105 04:57:41.864678  7168 replica.cpp:664] Replica learned APPEND action at position 8005
    I1105 04:57:41.866068  7171 registrar.cpp:488] Successfully updated the 'registry' in 7.5648ms
    I1105 04:57:41.866261  7168 log.cpp:704] Attempting to truncate the log to 8005
    I1105 04:57:41.866374  7173 coordinator.cpp:341] Coordinator attempting to write TRUNCATE action at position 8006
    I1105 04:57:41.866920  7169 replica.cpp:511] Replica received write request for position 8006
    I1105 04:57:41.868661  7174 master.cpp:3699] Registered slave 20151105-045733-3314083338-5050-7152-S0 at slave(1)@10.29.23.28:5051 (ip-10-29-23-28.ec2.internal) with cpus(*):8; mem(*):5986; disk(*):196338; ports(*):[31000-32000]
    I1105 04:57:41.868976  7169 leveldb.cpp:343] Persisting action (18 bytes) to leveldb took 1.996354ms
    I1105 04:57:41.869005  7169 replica.cpp:679] Persisted action at 8006
    I1105 04:57:41.869000  7170 hierarchical.hpp:540] Added slave 20151105-045733-3314083338-5050-7152-S0 (ip-10-29-23-28.ec2.internal) with cpus(*):8; mem(*):5986; disk(*):196338; ports(*):[31000-32000] (allocated: )
    I1105 04:57:41.869566  7174 replica.cpp:658] Replica received learned notice for position 8006
    I1105 04:57:41.870249  7174 leveldb.cpp:343] Persisting action (20 bytes) to leveldb took 633721ns
    I1105 04:57:41.870314  7174 leveldb.cpp:401] Deleting ~2 keys from leveldb took 32741ns
    I1105 04:57:41.870342  7174 replica.cpp:679] Persisted action at 8006
    I1105 04:57:41.870370  7174 replica.cpp:664] Replica learned TRUNCATE action at position 8006
    I1105 04:57:41.870602  7167 master.cpp:3998] Received update of slave 20151105-045733-3314083338-5050-7152-S0 at slave(1)@10.29.23.28:5051 (ip-10-29-23-28.ec2.internal) with total oversubscribed resources
    I1105 04:57:41.871006  7167 hierarchical.hpp:600] Slave 20151105-045733-3314083338-5050-7152-S0 (ip-10-29-23-28.ec2.internal) updated with oversubscribed resources  (total: cpus(*):8; mem(*):5986; disk(*):196338; ports(*):[31000-32000], allocated: )

    可以看到slave注册的时候(红色部分)把机器的全部资源都注册上了,可以通过选项来限制资源:

    --resources="mem:8096;cpus:2"

    框架跑起来比较容易,关健是如何使用起来。

  • 相关阅读:
    基于 JavaEE 的在线考试系统
    别在发愁写页面了,强烈推荐几款傻瓜式扒网站神器!!!
    JAVA WEB 婚纱摄影管理系统(影楼系统)
    JAVA WEB 文件管理系统
    基于SSM的洗车管理系统
    基于JAVA WEB 的健身平台系统
    JAVA WEB 宠物商店
    JAVA WEB 家政信息发布平台
    JAVA WEB 文件管理系统
    JAVA WEB 选课系统
  • 原文地址:https://www.cnblogs.com/zhangqingping/p/4939021.html
Copyright © 2020-2023  润新知