• zmq导致master收不到minion的key


    问题:

    官网方式yum安装完saltstack之后,master与minion都成功启动后,但是却发现怎么都收不到minion的key。(zmq版本不一致所致,困扰了我好几天,希望能帮到大家。)

    分析:

    master上的状态:

    # 查看master 上的配置文件,只修改了interface:
    [root@master doubles]# cat /etc/salt/master
    
    interface: 192.168.188.128
    
    [root@master doubles]# systemctl status salt-master
    ● salt-master.service - The Salt Master Server
       Loaded: loaded (/usr/lib/systemd/system/salt-master.service; disabled; vendor preset: disabled)
       Active: active (running) since Tue 2019-01-08 09:55:59 HKT; 7min ago
         Docs: man:salt-master(1)
               file:///usr/share/doc/salt/html/contents.html
               https://docs.saltstack.com/en/latest/contents.html
     Main PID: 9331 (salt-master)
       CGroup: /system.slice/salt-master.service
               ├─9331 /usr/bin/python /usr/bin/salt-master
               ├─9340 /usr/bin/python /usr/bin/salt-master
               ├─9345 /usr/bin/python /usr/bin/salt-master
               ├─9348 /usr/bin/python /usr/bin/salt-master
               ├─9349 /usr/bin/python /usr/bin/salt-master
               ├─9350 /usr/bin/python /usr/bin/salt-master
               ├─9351 /usr/bin/python /usr/bin/salt-master
               ├─9352 /usr/bin/python /usr/bin/salt-master
               ├─9353 /usr/bin/python /usr/bin/salt-master
               ├─9354 /usr/bin/python /usr/bin/salt-master
               ├─9361 /usr/bin/python /usr/bin/salt-master
               ├─9362 /usr/bin/python /usr/bin/salt-master
               └─9364 /usr/bin/python /usr/bin/salt-master
    
    Jan 08 09:55:54 master.doubles.cn systemd[1]: Starting The Salt Master Server...
    Jan 08 09:55:59 master.doubles.cn systemd[1]: Started The Salt Master Server.
    
    [root@master doubles]# salt-key -L
    Accepted Keys:
    Denied Keys:
    Unaccepted Keys:
    Rejected Keys:
    # 始终看不到minion的key。
    [root@master doubles]# salt-key -a 192.168.188.129
    The key glob '192.168.188.129' does not match any unaccepted keys.
    [root@master doubles]# salt '*' test.ping
    Salt request timed out. The master is not responding. You may need to run your command with `--async` in order to bypass the congested event bus. With `--async`, the CLI tool will print the job id (jid) and exit immediately without listening for responses. You can then use `salt-run jobs.lookup_jid` to look up the results of the job in the job cache later.
    

    看下minion上的salt状态:

    # 查看minion上的配置文件,发现只修改了master与id如下:
    [root@slave doubles]# vim /etc/salt/minion
    
    master: 192.168.188.128
    
    id: 192.168.188.129
    
    # salt-minion开启调试模式 
    [root@slave doubles]# salt-minion -l debug
    ...
    [DEBUG   ] SaltReqTimeoutError, retrying. (1/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (2/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (3/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (4/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (5/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (6/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (7/7)
    [DEBUG   ] Re-init ZMQ socket: Message timed out
    [DEBUG   ] Trying to connect to: tcp://192.168.188.128:4506
    [ERROR   ] Error while bringing up minion for multi-master. Is master at 192.168.188.128 responding?
    [DEBUG   ] Connecting to master. Attempt 1 of 1
    [DEBUG   ] Master URI: tcp://192.168.188.128:4506
    [DEBUG   ] Re-using AsyncAuth for (u'/etc/salt/pki/minion', u'192.168.188.129', u'tcp://192.168.188.128:4506')
    [DEBUG   ] Generated random reconnect delay between '1000ms' and '11000ms' (7456)
    [DEBUG   ] Setting zmq_reconnect_ivl to '7456ms'
    [DEBUG   ] Setting zmq_reconnect_ivl_max to '11000ms'
    [DEBUG   ] Re-using AsyncZeroMQReqChannel for (u'/etc/salt/pki/minion', u'192.168.188.129', u'tcp://192.168.188.128:4506', 'clear')
    [DEBUG   ] salt.crypt.get_rsa_pub_key: Loading public key
    [DEBUG   ] SaltReqTimeoutError, retrying. (1/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (2/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (3/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (4/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (5/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (6/7)
    [DEBUG   ] SaltReqTimeoutError, retrying. (7/7)
    [DEBUG   ] Re-init ZMQ socket: Message timed out
    [DEBUG   ] Trying to connect to: tcp://192.168.188.128:4506
    [ERROR   ] Error while bringing up minion for multi-master. Is master at 192.168.188.128 responding?
    ...
    

    在这里,本机上的master也是无法收到本机上的minion的key。找了好久,有的说是配置里的master与id没有配对,有的说是增大线程数,有的说是防火墙,有的说是删掉pki下的id,试了都没有用。最后,认真思考一下,其实是minion无法与master 通信,偶然对比了一下minion与master的版本:

    master上:

    [root@master doubles]# salt-master --versions-report
    Salt Version:
               Salt: 2018.3.3
     
    Dependency Versions:
               cffi: 1.6.0
           cherrypy: Not Installed
           dateutil: Not Installed
          docker-py: Not Installed
              gitdb: Not Installed
          gitpython: Not Installed
              ioflo: Not Installed
             Jinja2: 2.7.2
            libgit2: Not Installed
            libnacl: Not Installed
           M2Crypto: 0.21.1
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.5.6
       mysql-python: Not Installed
          pycparser: 2.14
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 2.7.5 (default, Oct 30 2018, 23:45:53)
       python-gnupg: Not Installed
             PyYAML: 3.10
              PyZMQ: 14.3.1
               RAET: Not Installed
              smmap: Not Installed
            timelib: Not Installed
            Tornado: 4.2.1
                ZMQ: 3.2.2
     
    System Versions:
               dist: centos 7.6.1810 Core
             locale: UTF-8
            machine: x86_64
            release: 3.10.0-957.1.3.el7.x86_64
             system: Linux
            version: CentOS Linux 7.6.1810 Core

    minion上:

    [root@slave doubles]# salt-minion --versions-report
    Salt Version:
               Salt: 2018.3.3
     
    Dependency Versions:
               cffi: Not Installed
           cherrypy: Not Installed
           dateutil: Not Installed
          docker-py: Not Installed
              gitdb: Not Installed
          gitpython: Not Installed
              ioflo: Not Installed
             Jinja2: 2.7.2
            libgit2: Not Installed
            libnacl: Not Installed
           M2Crypto: Not Installed
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.4.6
       mysql-python: Not Installed
          pycparser: Not Installed
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 2.7.5 (default, Oct 30 2018, 23:45:53)
       python-gnupg: Not Installed
             PyYAML: 3.11
              PyZMQ: 15.3.0
               RAET: Not Installed
              smmap: Not Installed
            timelib: Not Installed
            Tornado: 4.2.1
                ZMQ: 4.1.4
     
    System Versions:
               dist: centos 7.6.1810 Core
             locale: UTF-8
            machine: x86_64
            release: 3.10.0-957.1.3.el7.x86_64
             system: Linux
            version: CentOS Linux 7.6.1810 Core

    发现master与minion上的ZMQ版本不一样,这应该就是问题所在了。

    解决:

    尝试更新master上的zmq版本。

    master上:

    [root@master doubles]# yum install *zmq* –y
    # 再对比版本
    [root@master doubles]# salt-master --versions-report
    Salt Version:
               Salt: 2018.3.3
     
    Dependency Versions:
               cffi: 1.6.0
           cherrypy: Not Installed
           dateutil: Not Installed
          docker-py: Not Installed
              gitdb: Not Installed
          gitpython: Not Installed
              ioflo: Not Installed
             Jinja2: 2.7.2
            libgit2: Not Installed
            libnacl: Not Installed
           M2Crypto: 0.21.1
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.5.6
       mysql-python: Not Installed
          pycparser: 2.14
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 2.7.5 (default, Oct 30 2018, 23:45:53)
       python-gnupg: Not Installed
             PyYAML: 3.10
              PyZMQ: 15.3.0
               RAET: Not Installed
              smmap: Not Installed
            timelib: Not Installed
            Tornado: 4.2.1
                ZMQ: 4.1.4
     
    System Versions:
               dist: centos 7.6.1810 Core
             locale: UTF-8
            machine: x86_64
            release: 3.10.0-957.1.3.el7.x86_64
             system: Linux
            version: CentOS Linux 7.6.1810 Core

    此时发现zmq版本已经和minion一致了。重启master再试一下:

    master:

    [root@master doubles]# systemctl restart salt-master
    [root@master doubles]# systemctl status salt-master
    ● salt-master.service - The Salt Master Server
       Loaded: loaded (/usr/lib/systemd/system/salt-master.service; disabled; vendor preset: disabled)
       Active: active (running) since Tue 2019-01-08 11:01:43 HKT; 17s ago
         Docs: man:salt-master(1)
               file:///usr/share/doc/salt/html/contents.html
               https://docs.saltstack.com/en/latest/contents.html
     Main PID: 15436 (salt-master)
       CGroup: /system.slice/salt-master.service
               ├─15436 /usr/bin/python /usr/bin/salt-master
               ├─15445 /usr/bin/python /usr/bin/salt-master
               ├─15450 /usr/bin/python /usr/bin/salt-master
               ├─15453 /usr/bin/python /usr/bin/salt-master
               ├─15454 /usr/bin/python /usr/bin/salt-master
               ├─15455 /usr/bin/python /usr/bin/salt-master
               ├─15456 /usr/bin/python /usr/bin/salt-master
               ├─15457 /usr/bin/python /usr/bin/salt-master
               ├─15458 /usr/bin/python /usr/bin/salt-master
               ├─15459 /usr/bin/python /usr/bin/salt-master
               ├─15460 /usr/bin/python /usr/bin/salt-master
               ├─15461 /usr/bin/python /usr/bin/salt-master
               └─15462 /usr/bin/python /usr/bin/salt-master
    
    Jan 08 11:01:42 master.doubles.cn systemd[1]: Stopped The Salt Master Server.
    Jan 08 11:01:42 master.doubles.cn systemd[1]: Starting The Salt Master Server...
    Jan 08 11:01:43 master.doubles.cn systemd[1]: Started The Salt Master Server.

    minion上,重启:

    [root@slave doubles]# salt-minion -l debug
    ...
    [DEBUG   ] Trying to connect to: tcp://192.168.188.128:4506
    [DEBUG   ] Initializing new AsyncZeroMQReqChannel for (u'/etc/salt/pki/minion', u'192.168.188.129', u'tcp://192.168.188.128:4506', u'aes')
    [DEBUG   ] Initializing new AsyncAuth for (u'/etc/salt/pki/minion', u'192.168.188.129', u'tcp://192.168.188.128:4506')
    [DEBUG   ] Connecting the Minion to the Master URI (for the return server): tcp://192.168.188.128:4506
    [DEBUG   ] Trying to connect to: tcp://192.168.188.128:4506
    [INFO    ] Minion is ready to receive requests!
    [DEBUG   ] Minion of '192.168.188.128' is handling event tag '/salt/minion/minion_schedule_delete_complete'
    [DEBUG   ] Minion of '192.168.188.128' is handling event tag '/salt/minion/minion_schedule_add_complete'
    [DEBUG   ] Minion of '192.168.188.128' is handling event tag '/salt/minion/minion_schedule_delete_complete'
    [DEBUG   ] Minion of '192.168.188.128' is handling event tag '/salt/minion/minion_schedule_delete_complete'
    ...
    # 跟之前不一样了

    看下master上能否收到key了。

    master上:

    [root@master doubles]# salt-key -L
    Accepted Keys:
    Denied Keys:
    Unaccepted Keys:
    192.168.188.129
    Rejected Keys:
    [root@master doubles]# salt-key -a 192.168.188.129
    The following keys are going to be accepted:
    Unaccepted Keys:
    192.168.188.129
    Proceed? [n/Y] y
    Key for minion 192.168.188.129 accepted.
    [root@master doubles]# salt-key -L
    Accepted Keys:
    192.168.188.129
    Denied Keys:
    Unaccepted Keys:
    Rejected Keys:
    [root@master doubles]# salt "*" test.ping
    192.168.188.129:
        True

    发现已经OK了。

  • 相关阅读:
    个人总结13
    构建之法阅读笔记一
    个人总结12
    个人总结11
    地铁时光机第一阶段冲刺十
    地铁时光机第一阶段冲刺九
    地铁时光机第一阶段冲刺八
    单词 统计续
    地铁时光机第一阶段冲刺七
    个人总结10
  • 原文地址:https://www.cnblogs.com/doublexi/p/10238180.html
Copyright © 2020-2023  润新知