• centos 7 安装sql 审核工具 inception + archer


    系统环境: Centos7  + python2.7 + python3 .... 
    
    下载
    
    源码地址:https://github.com/mysql-inception/inception
    
    Inception安装
    
    1、安装相关依赖包
    
    yum install bison ncurses-libs libncurses5-dev ncurses-devel wget 
     git cmake openssl gcc-c++ -y
    
    注意:centos7 系统自带的bison的版本过高,在后面测试的时候会报错!
    下载地址: wget http://ftp.gnu.org/gnu/bison/bison-2.5.1.tar.gz
    
    2、安装inception
    
    git clone https://github.com/mysql-inception/inception
    cd inception
    sh inception_build.sh debug
    
    debug就是编译目录,编译后所有的生成文件都在这个目录下面,包括可执行文件Inception。可执行文件在debug/sql目录下面:
    
    [root@mysql sql]# pwd
    /root/inception/debug/sql
    [root@mysql sql]# ls
    CMakeFiles           dummy.bak     Inception   libbinlog.a  librpl.a    libsql.a          Makefile  sql_builtin.cc  sql_yacc.h
    cmake_install.cmake  gen_lex_hash  lex_hash.h  libmaster.a  libslave.a  libsqlgunitlib.a  share     sql_yacc.cc
    [root@mysql3 sql]#
    
    3、创建配置目录,将可执行文件Inception拷贝到/opt/inception/bin目录
    [root@mysql sql]# mkdir -p /data/inception
    [root@mysql sql]# mkdir -p /data/inception/bin
    [root@mysql sql]# mkdir -p /data/inception/temp
    [root@mysql sql]# cp Inception /data/inception/bin/
    编辑配置文件:
    
    [root@mysql sql]# cat /etc/inc.cnf
    [inception]
    general_log=1
    general_log_file=/opt/inception/inception.log
    port=6669
    socket=/opt/inception/inc.socket
    character-set-client-handshake=0
    character-set-server=utf8
    inception_remote_system_password=123456
    inception_remote_system_user=root
    inception_remote_backup_port=3306
    inception_remote_backup_host=127.0.0.1
    inception_support_charset=utf8mb4
    inception_enable_nullable=0
    inception_check_primary_key=1
    inception_check_column_comment=1
    inception_check_table_comment=1
    inception_osc_min_table_size=1
    inception_osc_bin_dir=/opt/inception/temp
    inception_osc_chunk_time=0.1
    inception_ddl_support=1
    inception_enable_blob_type=1
    inception_check_column_default_value=1
    
    启动
    
    [root@mysql sql]# nohup /data/inception/bin/Inception --defaults-file=/etc/inc.cnf &
    [1] 2398
    [root@mysql sql]# nohup: ignoring input and appending output to `nohup.out'
    [root@mysql sql]# ps -ef|grep Inception
    root      2414 23520  0 23:15 pts/0    00:00:00 /opt/inception/bin/Inception --defaults-file=/etc/inc.cnf
    root      2419 23520  0 23:22 pts/0    00:00:00 grep Inception
    
    登录验证
    
    [root@mysql sql]# mysql -uroot -h127.0.0.1 -P6669
    Welcome to the MySQL monitor.  Commands end with ; or g.
    Your MySQL connection id is 1
    Server version: Inception2.1.50 1
    Copyright (c) 2009-2014 Percona LLC and/or its affiliates
    Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
    Oracle is a registered trademark of Oracle Corporation and/or its
    affiliates. Other names may be trademarks of their respective
    owners.
    Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
    mysql> inception get variables;
    +------------------------------------------+-------------------------------------------------+
    | Variable_name                            | Value                                           |
    +------------------------------------------+-------------------------------------------------+
    | autocommit                               | OFF                                             |
    | bind_address                             | *                                               |
    | character_set_system                     | utf8                                            |
    | character_sets_dir                       | /src/inception-master/debug/sql/share/charsets/ |
    | connect_timeout                          | 10                                              |
    | date_format                              | %Y-%m-%d                                        |
    | datetime_format                          | %Y-%m-%d %H:%i:%s                               |
    | general_log                              | ON                                              |
    | general_log_file                         | /opt/inception/inception.log                    |
    | inception_check_autoincrement_datatype   | ON                                              |
    | inception_check_autoincrement_init_value | ON                                              |
    | inception_check_autoincrement_name       | ON                                              |
    | inception_check_column_comment           | ON                                              |
    | inception_check_column_default_value     | ON                                              |
    | inception_check_dml_limit                | ON                                              |
    | inception_check_dml_orderby              | ON                                              |
    | inception_check_dml_where                | ON                                              |
    | inception_check_identifier               | ON                                              |
    | inception_check_index_prefix             | ON                                              |
    | inception_check_insert_field             | ON                                              |
    | inception_check_primary_key              | ON                                              |
    | inception_check_table_comment            | ON                                              |
    | inception_check_timestamp_default        | ON                                              |
    | inception_ddl_support                    | ON                                              |
    | inception_enable_autoincrement_unsigned  | ON                                              |
    | inception_enable_blob_type               | ON                                              |
    | inception_enable_column_charset          | OFF                                             |
    | inception_enable_enum_set_bit            | OFF                                             |
    | inception_enable_foreign_key             | OFF                                             |
    | inception_enable_identifer_keyword       | OFF                                             |
    | inception_enable_not_innodb              | OFF                                             |
    | inception_enable_nullable                | OFF                                             |
    | inception_enable_orderby_rand            | OFF                                             |
    | inception_enable_partition_table         | OFF                                             |
    | inception_enable_pk_columns_only_int     | OFF                                             |
    | inception_enable_select_star             | OFF                                             |
    | inception_enable_sql_statistic           | ON                                              |
    | inception_max_char_length                | 16                                              |
    | inception_max_key_parts                  | 5                                               |
    | inception_max_keys                       | 16                                              |
    | inception_max_primary_key_parts          | 5                                               |
    | inception_max_update_rows                | 10000                                           |
    | inception_merge_alter_table              | ON                                              |
    | inception_osc_alter_foreign_keys_method  | none                                            |
    | inception_osc_bin_dir                    | /opt/inception/temp                             |
    | inception_osc_check_alter                | ON                                              |
    | inception_osc_check_interval             | 5.000000                                        |
    | inception_osc_check_replication_filters  | ON                                              |
    | inception_osc_chunk_size                 | 1000                                            |
    | inception_osc_chunk_size_limit           | 4.000000                                        |
    | inception_osc_chunk_time                 | 0.100000                                        |
    | inception_osc_critical_thread_connected  | 1000                                            |
    | inception_osc_critical_thread_running    | 80                                              |
    | inception_osc_drop_new_table             | ON                                              |
    | inception_osc_drop_old_table             | ON                                              |
    | inception_osc_max_lag                    | 3.000000                                        |
    | inception_osc_max_thread_connected       | 1000                                            |
    | inception_osc_max_thread_running         | 80                                              |
    | inception_osc_min_table_size             | 1                                               |
    | inception_osc_on                         | ON                                              |
    | inception_osc_print_none                 | ON                                              |
    | inception_osc_print_sql                  | ON                                              |
    | inception_osc_recursion_method           | processlist                                     |
    | inception_password                       |                                                 |
    | inception_read_only                      | OFF                                             |
    | inception_remote_backup_host             | 127.0.0.1                                       |
    | inception_remote_backup_port             | 3306                                            |
    | inception_remote_system_password         | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9       |
    | inception_remote_system_user             | root                                            |
    | inception_support_charset                | utf8mb4                                         |
    | inception_user                           |                                                 |
    | interactive_timeout                      | 28800                                           |
    | max_allowed_packet                       | 1073741824                                      |
    | max_connect_errors                       | 100                                             |
    | max_connections                          | 151                                             |
    | net_buffer_length                        | 16384                                           |
    | net_read_timeout                         | 30                                              |
    | net_write_timeout                        | 60                                              |
    | port                                     | 6669                                            |
    | query_alloc_block_size                   | 8192                                            |
    | query_prealloc_size                      | 8192                                            |
    | socket                                   | /opt/inception/inc.socket                       |
    | thread_handling                          | one-thread-per-connection                       |
    | thread_stack                             | 262144                                          |
    | time_format                              | %H:%i:%s                                        |
    | version                                  | Inception2.1.50                                 |
    | version_comment                          | Source distribution                             |
    | version_compile_machine                  | x86_64                                          |
    | version_compile_os                       | Linux                                           |
    | wait_timeout                             | 28800                                           |
    +------------------------------------------+-------------------------------------------------+
    90 rows in set (0.00 sec)
    mysql> exit
    Bye
    [root@mysql sql]#
    
    可以看到所有的变量,安装成功!
    使用Inception
    DDL审核
    
    [root@mysql ~]# cat inception.py 
    #!/usr/bin/python
    #-*-coding: utf-8-*-
    import pymysql
    sql='/*--user=testdev;--password=testdev;--host=127.0.0.1;--execute=1;--port=3306;*/
    inception_magic_start;
    use testdb;
    CREATE TABLE adaptive_office(id int);
    inception_magic_commit;'
    try:
        conn=pymysql.connect(host='127.0.0.1',user='root',password='123456',db='',port=6669)
        cur=conn.cursor()
        ret=cur.execute(sql)
        result=cur.fetchall()
        num_fields = len(cur.description) 
        field_names = [i[0] for i in cur.description]
        print field_names
        for row in result:
            print row[0], "|",row[1],"|",row[2],"|",row[3],"|",row[4],"|",
            row[5],"|",row[6],"|",row[7],"|",row[8],"|",row[9],"|",row[10]
        cur.close()
        conn.close()
    except pymysql.Error,e:
         print "Mysql Error %d: %s" % (e.args[0], e.args[1])
    [root@mysql ~]#
    
    运行结果:
    
    [root@mysql ~]# python inception.py 
    [u'ID', u'stage', u'errlevel', u'stagestatus', u'errormessage', u'SQL', u'Affected_rows', u'sequence', u'backup_dbname', u'execute_time', u'sqlsha1']
    1 | CHECKED | 0 | Audit completed | None | 2 | CHECKED | 1 | Audit completed | Set engine to innodb for table 'adaptive_office'.
    Set charset to one of 'utf8mb4' for table 'adaptive_office'.
    Set comments for table 'adaptive_office'.
    Column 'id' in table 'adaptive_office' have no comments.
    Column 'id' in table 'adaptive_office' is not allowed to been nullable.
    Set Default value for column 'id' in table 'adaptive_office'
    Set a primary key for table 'adaptive_office'. |
    
    常见错误及解决
    [root@mysql ~]# python inception.py 
    Traceback (most recent call last):
      File "inception.py", line 10, in <module>
        conn=pymysql.connect(host='192.168.1.123',user='root',password='123456',db='',port=6669)
      File "/usr/local/lib/python2.7/site-packages/pymysql/__init__.py", line 88, in Connect
        return Connection(*args, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 679, in __init__
        self.connect()
      File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 891, in connect
        self._request_authentication()
      File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 1054, in _request_authentication
        if int(self.server_version.split('.', 1)[0]) >= 5:
    ValueError: invalid literal for int() with base 10: 'Inception2'
    
    
    解决办法:
    [root@mysql ~]# vi /usr/local/lib/python2.7/site-packages/pymysql/connections.py
    ... ...
        def _request_authentication(self):
            # https://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::HandshakeResponse
            self.server_version = '5.6.24-72.2-log'  #添加此行
            if int(self.server_version.split('.', 1)[0]) >= 5:
    ... ...
    
    使用中的其他错误见:
    https://riverdba.github.io/2017/04/13/inception-install/#
    
    
    命令行不是很友好,追求web界面的请继续,接下来安装archer
    
    我的安装环境:
    python3 + mysql5.7 + django1.8
    
    安装python3
    wget https://www.python.org/ftp/python/3.4.1/Python-3.4.1.tgz
    tar -zxvf Python-3.4.1.tgz   
    cd Python-3.4.1  
    ./configure  && make && make install 
    
    安装setuptools
    
    [root@mysql ~]# wget --no-check-certificate  https://pypi.python.org/packages/source/s/setuptools/setuptools-19.6.tar.gz#md5=c607dd118eae682c44ed146367a17e26
    [root@mysql ~]# tar -zxvf setuptools-19.6.tar.gz 
    [root@mysql ~]# cd setuptools-19.6
    [root@mysql setuptools-19.6]# python3 setup.py build
    [root@mysql setuptools-19.6]# python3 setup.py install
    ... ...
    creating 'dist/setuptools-19.6-py3.4.egg' and adding 'build/bdist.linux-x86_64/egg' to it
    removing 'build/bdist.linux-x86_64/egg' (and everything under it)
    Processing setuptools-19.6-py3.4.egg
    Copying setuptools-19.6-py3.4.egg to /opt/python3/lib/python3.4/site-packages
    Adding setuptools 19.6 to easy-install.pth file
    Installing easy_install script to /opt/python3/bin
    Installing easy_install-3.4 script to /opt/python3/bin
    Installed /opt/python3/lib/python3.4/site-packages/setuptools-19.6-py3.4.egg
    Processing dependencies for setuptools==19.6
    Finished processing dependencies for setuptools==19.6
    
    
    安装pip
    
    [root@mysql ~]# wget --no-check-certificate  https://pypi.python.org/packages/source/p/pip/pip-8.0.2.tar.gz#md5=3a73c4188f8dbad6a1e6f6d44d117eeb
    [root@mysql ~]# tar -zxvf pip-8.0.2.tar.gz 
    [root@mysql ~]# cd pip-8.0.2
    [root@mysql pip-8.0.2]# python3 setup.py build
    [root@mysql pip-8.0.2]# python3 setup.py install
    ... ...
    creating 'dist/pip-8.0.2-py3.4.egg' and adding 'build/bdist.linux-x86_64/egg' to it
    removing 'build/bdist.linux-x86_64/egg' (and everything under it)
    Processing pip-8.0.2-py3.4.egg
    creating /opt/python3/lib/python3.4/site-packages/pip-8.0.2-py3.4.egg
    Extracting pip-8.0.2-py3.4.egg to /opt/python3/lib/python3.4/site-packages
    Adding pip 8.0.2 to easy-install.pth file
    Installing pip3 script to /opt/python3/bin
    Installing pip3.4 script to /opt/python3/bin
    Installing pip script to /opt/python3/bin
    Installed /opt/python3/lib/python3.4/site-packages/pip-8.0.2-py3.4.egg
    Processing dependencies for pip==8.0.2
    Finished processing dependencies for pip==8.0.2
    
    安装Django
    
    使用pip3安装:pip3 install Django==1.8.17
    
    
    安装Crypto和pymysql
    
    [root@mysql3 ~]# pip3 install Crypto
    [root@mysql3 ~]# pip3 install pycrypto
    [root@mysql3 ~]# pip3 install pymysql
    [root@mysql3 ~]# vi /opt/python3/lib/python3.4/site-packages/pymysql/connections.py
    在if int(self.server_version.split(‘.’, 1)[0]) >= 5: 这一行之前加上以下这一句并保存,记得别用tab键用4个空格缩进:
    self.server_version = '5.6.24-72.2-log'
    修改后如下图:
    
    
    
    配置archer
    
    1、下载archer
    
    [root@mysql ~]# mkdir -p /data/
    [root@mysql ~]# cd /data
    [root@mysql data]# git clone https://github.com/jly8866/archer.git
    [root@mysql data]# ls archer/*
    archer  debug.sh  docs  manage.py  README.md  requirements.txt  screenshots  sql  startup.sh  stop.sh
    
    2、创建archer DB和inception DB
    
    root@(none) 04:04:54>create database archer;
    Query OK, 1 row affected (0.00 sec)
    root@(none) 04:05:12>grant all on archer.* to archer_rw@'%' identified by 'archer_rw';
    Query OK, 0 rows affected (0.03 sec)
    root@(none) 04:06:42>flush privileges;
    Query OK, 0 rows affected (0.02 sec)
    root@(none) 04:13:39>create database inception_db;
    Query OK, 1 row affected (0.00 sec)
    root@(none) 04:13:46>grant all on inception_db.* to incep_rw@'%' identified by 'incep_rw';                    
    Query OK, 0 rows affected (0.00 sec)
    root@(none) 04:14:23>flush privileges;
    Query OK, 0 rows affected (0.00 sec)
    
    3、修改/data/archer/settings.py,如下内容:
    
    #该项目本身的mysql数据库地址
    DATABASES = {
        'default': {
            'ENGINE': 'django.db.backends.mysql',
            'NAME': 'archer',
            'USER': 'archer_rw',
            'PASSWORD': 'archer_rw',
            'HOST': '127.0.0.1',
            'PORT': '3306'
        }
    }
    #inception组件所在的地址
    INCEPTION_HOST = '127.0.0.1'
    INCEPTION_PORT = '6669'
    #查看回滚SQL时候会用到,这里要告诉archer去哪个mysql里读取inception备份的回滚信息和SQL.
    #注意这里要和inception组件的inception.conf里的inception_remote_XX部分保持一致.
    INCEPTION_REMOTE_BACKUP_HOST='127.0.0.1'
    INCEPTION_REMOTE_BACKUP_PORT=3306
    INCEPTION_REMOTE_BACKUP_USER='incep_rw'
    INCEPTION_REMOTE_BACKUP_PASSWORD='incep_rw'
    
    4、通过model创建archer本身的数据库表
    
    [root@mysql]# cd /data/archer
    [root@mysql archer]# python3 manage.py makemigrations
    No changes detected
    [root@mysql archer]# python3 manage.py makemigrations sql
    Migrations for 'sql':
      0001_initial.py:
        - Create model users
        - Create model master_config
        - Create model workflow
    [root@mysql archer]# python3 manage.py migrate
    Operations to perform:
      Synchronize unmigrated apps: messages, staticfiles
      Apply all migrations: sessions, auth, contenttypes, sql, admin
    Synchronizing apps without migrations:
      Creating tables...
        Running deferred SQL...
      Installing custom SQL...
    Running migrations:
      Rendering model states... DONE
      Applying contenttypes.0001_initial... OK
      Applying contenttypes.0002_remove_content_type_name... OK
      Applying auth.0001_initial... OK
      Applying auth.0002_alter_permission_name_max_length... OK
      Applying auth.0003_alter_user_email_max_length... OK
      Applying auth.0004_alter_user_username_opts... OK
      Applying auth.0005_alter_user_last_login_null... OK
      Applying auth.0006_require_contenttypes_0002... OK
      Applying sql.0001_initial... OK
      Applying admin.0001_initial... OK
      Applying sessions.0001_initial... OK
    查看表是否创建成功:  
    [root@mysql archer]# mysql -uarcher_rw -parcher_rw -h127.0.0.1 -P3306 -Darcher -e "show tables;"
    Warning: Using a password on the command line interface can be insecure.
    +----------------------------+
    | Tables_in_archer           |
    +----------------------------+
    | auth_group                 |
    | auth_group_permissions     |
    | auth_permission            |
    | django_admin_log           |
    | django_content_type        |
    | django_migrations          |
    | django_session             |
    | sql_master_config          |
    | sql_users                  |
    | sql_users_groups           |
    | sql_users_user_permissions |
    | sql_workflow               |
    +----------------------------+
    
    5、创建django admin管理员
    
    [root@mysql archer]# python3 manage.py createsuperuser
    Username: admin
    Email address: admin@foxmail.com
    Password: admin
    Password (again): admin
    Superuser created successfully.
    该用户可以登录django admin来管理model。
    
    启动acher
    
    [root@mysql archer]# vi debug.sh 
    #!/bin/bash
    python3 manage.py runserver 0.0.0.0:9123
    [root@mysql3 archer]# sh debug.sh 
    Performing system checks...
    System check identified no issues (0 silenced).
    April 15, 2017 - 16:39:11
    Django version 1.8.17, using settings 'archer.settings'
    Starting development server at http://0.0.0.0:9123/
    Quit the server with CONTROL-C.
    
    
    注意:
        1.如果有RuntimeError: Compression requires the (missing) zlib module  报错
        
        解决办法:
        yum install zlib -y
    
        yum install zlib-devel -y
    
        安装完成后,重新编译 python3 make && make install 就ok!!
    
        2.如果有AttributeError: 'module' object has no attribute 'HTTPSConnection' 报错
    
        解决办法:        
        yum -y install openssl openssl-devel
    
        安装完成后,重新编译 python3 make && make install 就ok!!
    剩下的就是web界面配置
      链接:https://riverdba.github.io/2017/04/15/archer-install/ 如果有什么安装问题,可以加我qq:1305464670 一起学习,一起进步!
  • 相关阅读:
    查询快递单号-宅急送快递接口
    对路由转发的浅显理解
    对SpringCloud Zuul 的基本使用总结
    对SpringCloud Hystrix的使用个人总结
    对SpringBoot开箱即用使用方法的浅显理解
    对SpringCloud Hystrix服务降级的浅显理解
    金玉良言
    2种运行时织入的动态代理模式原理和调用流程比较
    记录手动启动Oracle数据库的方式
    基于SpringBoot搭建一个带数据库访问的WEB项目(记录所需的依赖,配置,驱动安装等注意事项)
  • 原文地址:https://www.cnblogs.com/chenjiaxin--007/p/8432795.html
Copyright © 2020-2023  润新知