1.clickhouse集群部署
特点:
1. 依赖ZooKeeper,通过其来协调多个副本之间的同步。
2. 表级别的副本,副本是在表级别定义的。
3. 多主架构,可以在任意副本上执行语句。
部署规划 10台机器 ,5个分片, 2个副本
集群机器ip列表:
192.168.10.101
192.168.10.102
192.168.10.103
192.168.10.104
192.168.10.105
192.168.10.106
192.168.10.107
192.168.10.108
192.168.10.109
192.168.10.110
2.配置/etc/hosts
在所有clickhouse 集群机器上添加主机名和ip地址映射关系。
192.168.10.101 clickhouse1
192.168.10.102 clickhouse2
192.168.10.103 clickhouse3
192.168.10.104 clickhouse4
192.168.10.105 clickhouse5
192.168.10.106 clickhouse6
192.168.10.107 clickhouse7
192.168.10.108 clickhouse8
192.168.10.109 clickhouse9
192.168.10.110 clickhouse10
3.安装配置zookeeper
参考该步骤
https://www.cnblogs.com/whiteY/p/14898503.html
4.下载clickhouse安装包
clickhouse-client-21.9.7.2-2.noarch.rpm
clickhouse-common-static-21.9.7.2-2.x86_64.rpm
clickhouse-common-static-dbg-21.9.7.2-2.x86_64.rpm
clickhouse-server-21.9.7.2-2.noarch.rpm
下载地址:
https://repo.yandex.ru/clickhouse/rpm/stable/x86_64/
!!!部署集群如果有httpd服务则只需要将安装包及修改后的配置文件放在下载服务器上,在需要安装服务的机器进行分发下载配置,没有的话则忽略该内容直接看第5步!!!
httpd下载服务器rpm安装包及修改配置自动化脚本文档待补充
!!!部署集群如果有httpd服务则只需要将安装包及修改后的配置文件放在下载服务器上,在需要安装服务的机器进行分发下载配置,没有的话则忽略该内容直接看第5步!!!
5.在所有安装集群节点安装rpm包
rpm -ivh *.rpm
安装过程中需要设置默认用户密码 :123456
Enter password for default user:
6.在所有安装集群节点下配置clickhouse服务的配置文件config.xml 和 users.xml。
vi config.xml
下边只列出需要修改的内容
在所有机器节点修改一下listen_host为本机业务ip,
允许IP4和IP6源主机远程访问
<listen_host>::</listen_host>
192.168.10.101
<listen_host>::</listen_host>
<listen_host>192.168.10.101</listen_host>
192.168.10.102
<listen_host>::</listen_host>
<listen_host>192.168.10.102</listen_host>
以此类推
修改时区
<timezone>Asia/Shanghai</timezone>
设置分片副本及zookeeper配置,5个分片2个副本,所有集群该配置都一样
shard:分片
replica:副本
<internal_replication>true</internal_replication> //添加这一句,可减少distributed压力,用已有的复制表引擎分发同步数据
<remote_servers>
<!-- Test only shard config for testing distributed storage -->
<cluster_5_shards_2_replica>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>192.168.10.101</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
<replica>
<host>192.168.10.102</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>192.168.10.103</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
<replica>
<host>192.168.10.104</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>192.168.10.105</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
<replica>
<host>192.168.10.106</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>192.168.10.107</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
<replica>
<host>192.168.10.108</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
</shard>
<shard>
<internal_replication>true</internal_replication>
<replica>
<host>192.168.10.109</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
<replica>
<host>192.168.10.110</host>
<port>9000</port>
<user>default</user>
<password>123456</password>
</replica>
</shard>
</cluster_5_shards_2_replica>
</remote_servers>
<zookeeper>
<node>
<host>192.168.10.111</host>
<port>2181</port>
</node>
<node>
<host>192.168.10.112</host>
<port>2181</port>
</node>
<node>
<host>192.168.10.113</host>
<port>2181</port>
</node>
</zookeeper>
修改macros,不同机器编号不同
macros 相当于是每个cliclhouse的环境变量。定义macros的作用在于可以用同一个包含macros变量的 SQL在所有的 clickhouse-server节点上执行。
192.168.10.101
<macros>
<shard>1</shard>
<replica>01</replica>
</macros>
192.168.10.102
<macros>
<shard>1</shard>
<replica>02</replica>
</macros>
192.168.10.103
<macros>
<shard>2</shard>
<replica>01</replica>
</macros>
192.168.10.104
<macros>
<shard>2</shard>
<replica>02</replica>
</macros>
......以此类推5个分片2副本
clickhouse默认初始化数据目录在/var/lib/clickhouse/,如果想指定数据目录到data下则需要修改配置文件里边的数据目录
<path>/data/clickhouse/</path>
<tmp_path>/data/clickhouse/tmp/</tmp_path>
<user_files_path>/data/clickhouse/user_files/</user_files_path>
<path>/data/clickhouse/access/</path>
<format_schema_path>/data/clickhouse/format_schemas/</format_schema_path>
mkdir -p /data/clickhouse
chown -R clickhouse:clickhouse /data/clickhouse
在所有机器节点修改users.xml添加用户
vi users.xml
添加用户test
<users>
<test>
<password_sha256_hex>5e8ffa57ce30c94e534b2813d9166f2acf8f8c8784efc91675b480021ecfc339</password_sha256_hex>
<networks>
<ip>::/0</ip>
</networks>
<!-- Settings profile for user. -->
<profile>test</profile>
<!-- Quota for user. -->
<quota>test</quota>
<!-- User can create other users and grant rights to them. -->
<access_management>1</access_management>
</test>
</users>
设置密码可以使用明文
<password>123456</password>
也可以使用密文sha256sum
<password_sha256_hex>8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92</password_sha256_hex>
加密方法为:
echo -n "123456" | sha256sum | tr -d '-'
6. 启动所有节点clickhouse服务
systemctl start clickhouse-server.service
7.登录clickhouse客户端查看集群
clickhouse-client -h 10.209.77.47 -u test --password
clickhouse-client -h 10.209.77.47 -u default --password
查询集群信息
select * from system.clusters;
至此集群部署完成。
参考链接
https://blog.csdn.net/tototuzuoquan/article/details/111027342