• Greenplum安装总结


    Greenplum安装总结

    一、环境说明

    服务器centos7 4台,一台Master节点,三台Segment节点:

    mdw 192.168.43.21 (master节点)

    sdw1 192.168.43.22   (segment节点,包含两个primary segment,两个mirror segment)

    sdw2  192.168.43.23   (segment节点,包含两个primary segment,两个mirror segment)

    sdw3  192.168.43.24   (segment节点,包含两个primary segment,两个mirror segment)

    GP版本: greenplum-db-5.7.1-rhel7-x86_64

    安装包下载(官网):https://network.pivotal.io/products/pivotal-gpdb/

    源码都在github上,也可自行编译源码安装:https://github.com/greenplum-db/gpdb

    二、安装前服务器准备

    (1) 关闭防火墙

    启动: systemctl start firewalld

    关闭: systemctl stop firewalld

    查看状态: systemctl status firewalld 

    开机禁用  : systemctl disable firewalld

    开机启用  : systemctl enable firewalld

    (2)配置hosts

    /ect/hosts:

    192.168.43.21   mdw

    192.168.43.22   sdw1

    192.168.43.23   sdw2

    192.168.43.24   sdw3

    确保所有机器ping所有的hostname都是通的

    (3)修改内核

    /etc/sysctl.conf 中追加

    kernel.shmmax = 500000000
    kernel.shmmni = 4096
    kernel.shmall = 4000000000
    kernel.sem = 500 1024000 200 4096
    kernel.sysrq = 1
    kernel.core_uses_pid = 1
    kernel.msgmnb = 65536
    kernel.msgmax = 65536
    kernel.msgmni = 2048
    net.ipv4.tcp_syncookies = 1
    net.ipv4.conf.default.accept_source_route = 0
    net.ipv4.tcp_tw_recycle = 1
    net.ipv4.tcp_max_syn_backlog = 4096
    net.ipv4.conf.all.arp_filter = 1
    net.ipv4.ip_local_port_range = 10000 65535
    net.core.netdev_max_backlog = 10000
    net.core.rmem_max = 2097152
    net.core.wmem_max = 2097152
    vm.overcommit_memory = 2
    vm.swappiness = 10
    vm.dirty_expire_centisecs = 500
    vm.dirty_writeback_centisecs = 100
    vm.dirty_background_ratio = 0
    vm.dirty_ratio=0
    vm.dirty_background_bytes = 1610612736
    vm.dirty_bytes = 4294967296

    sysctl -p

    (4)修改文件打开限制

    vi /etc/security/limits.conf

    * soft nofile 65536
    * hard nofile 65536
    * soft nproc 131072
    * hard nproc 131072

    (5)创建用户和用户组

    groupadd -g 530 gpadmin
    useradd -g 530 -u530 -m -d /home/gpadmin -s /bin/bash gpadmin
    chown -R gpadmin:gpadmin /home/gpadmin/
    passwd gpadmin

    (6)创建安装目录,并赋权限

    mkdir /opt/greenplum
    chown -R gpadmin:gpadmin /opt/greenplum/

    三、安装

    (1) 上传安装文件到主节点

    greenplum-db-5.7.1-rhel7-x86_64.zip 上传到/home/pgadmin 下并解压

    (2)安装

    ./greenplum-db-5.7.1-rhel7-x86_64.bin

    (安装过程选择安装目录:/opt/greenplum/greenplum-db-5.7.1/)

    安装成功后如下:

    安装完成后应用下环境变量:

    source /opt/greenplum/greenplum-db/greenplum_path.sh

    (3)配置hostlist文件记录所有节点,seg_hosts文件只记录segment节点

    [gpadmin@mdw ~]$ cat hostlist
    mdw
    sdw1
    sdw2
    sdw3
     
    [gpadmin@mdw ~]$ cat seg_hosts
    sdw1
    sdw2
    sdw3

    (4)gpssh-exkeys 打通服务器,即服务器间免密码登陆

    gpssh-exkeys -f hostlist

    打通后就可以使用gpssh命令对所有节点进行批量操作:

    [gpadmin@mdw ~]$ gpssh -f hostlist
    => pwd
    [sdw3] /home/gpadmin
    [sdw1] /home/gpadmin
    [sdw2] /home/gpadmin
    [ mdw] /home/gpadmin

    (5)把主节点安装好的程序压缩分发到每个子节点

    tar -cf gp5.7.tar greenplum-db-5.7.1/ #压缩
    gpscp -f /home/gpadmin/hostlist gp5.7.tar =:/opt/greenplum/ #分发

    (6)批量解压文件

    => tar -xf gp5.7.tar
    =>ln -s /opt/greenplum/greenplum-db-5.7.1 /opt/greenplum/greenplum-db #建立软链接

    (7)创建数据库数据目录

    #MASTER
    => mkdir -p /home/gpadmin/gpdata/gpmaster
    #PRIMARY
    => mkdir -p /home/gpadmin/gpdata/gpdatap1
    => mkdir -p /home/gpadmin/gpdata/gpdatap2
    #MIRROR
    => mkdir -p /home/gpadmin/gpdata/gpdatam1
    => mkdir -p /home/gpadmin/gpdata/gpdatam2

    (8)配置 .bash_profile 环境变量

    source /opt/greenplum/greenplum-db/greenplum_path.sh
    export MASTER_DATA_DIRECTORY=/home/gpadmin/gpdata/gpmaster/gpseg-1
    export PGPORT=5432
    export PGDATABASE=testDB #默认进入的db
    source .bash_profile

    (9)配置初始化配置文件,模板:/opt/greenplum/greenplum-db/docs/cli_help/gpconfigs/gpinitsystem_config

    可以根据模板修改,我的gpinit_config如下:

    ARRAY_NAME="Greenplum"
    SEG_PREFIX=gpseg
    PORT_BASE=40000
    declare -a DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatap1 /home/gpadmin/gpdata/gpdatap2)
    MASTER_HOSTNAME=mdw
    MASTER_DIRECTORY=/home/gpadmin/gpdata/gpmaster
    MASTER_PORT=5432
    TRUSTED_SHELL=/usr/bin/ssh
    CHECK_POINT_SEGMENTS=8
    ENCODING=UNICODE
    MIRROR_PORT_BASE=50000
    REPLICATION_PORT_BASE=41000
    MIRROR_REPLICATION_PORT_BASE=51000
    declare -a MIRROR_DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatam1 /home/gpadmin/gpdata/gpdatam2)
    MACHINE_LIST_FILE=/home/gpadmin/seg_hosts

    (10)初始化数据库

    gpinitsystem -c /home/gpadmin/gpinit_config -s sdw3

    其中sdw3是指master的standby(备份)所在的节点,书上和网上的一些资料都将standby放在最后一个节点,可能是约定俗成

    初始化根据脚本提示操作即可,如果上面有一些配置有问题,gpinitsystem就不能成功,日志在/home/gpadmin/gpAdminLogs中,认真查看日志,一味重复安装无意义

    (11)测试验证数据库

    登陆默认数据库postgres

    [gpadmin@mdw gpAdminLogs]$  psql -d postgres
    psql (8.3.23)
    Type "help" for help.
     
    postgres=#

    至此GP数据库集群就安装成功了

  • 相关阅读:
    Solr部署准备
    linux 安装SVN
    构建高性能web站点--读书大纲
    webapi请求
    搭建git服务器
    git常用知识整理
    Ruby on Rail 开发入门
    Ruby on rail 开发准备
    PHP模拟请求雅虎竞标商品加价
    JAVA httpURLConnection curl
  • 原文地址:https://www.cnblogs.com/jnba/p/10528220.html
Copyright © 2020-2023  润新知