logstash-input-jdbc/elasticsearch-jdbc缺点：删除记录没有办法同步，只能两边执行自己的删除命令，版本16年后未更新。
go-mysql-elasticsearch缺点：国内的不是很稳定
elasticsearch-river-MYSQL 版本太旧，12年之后未更新。
es安装：
下载
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.6.zip
解压
unzip elasticsearch-5.6.6.zip
移动到/usr/local目录
mv elasticsearch-5.6.6 /usr/local/elsearch
按需求修改config/elasticsearch.yml文件
vim /usr/local/elsearch/config/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /usr/local/elsearch/data
#
# Path to log files:
#
path.logs: /usr/local/elsearch/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 172.21.0.15
#
# Set a custom port for HTTP:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 3
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
http.cors.enabled: true
http.cors.allow-origin: "*"
安装中文分词设置
cd /usr/local/elsearch/
./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.6.6/elasticsearch-analysis-ik-5.6.6.zip
测试
curl -X PUT 'localhost:9200/accounts' -d '
{
"mappings": {
"person": {
"properties": {
"user": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_max_word"
},
"title": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_max_word"
},
"desc": {
"type": "text",
"analyzer": "ik_max_word",
"search_analyzer": "ik_max_word"
}
}
}
}
}'
参考链接 http://www.ruanyifeng.com/blog/2017/08/elasticsearch.html
es-head安装
下载
wget https://github.com/mobz/elasticsearch-head/archive/master.zip
解压
unzip master.zip
放到/usr/local/下
mv elasticsearch-head-master /usr/local/elsearch-head
安装node
于head插件本质上还是一个nodejs的工程，因此需要安装node，使用npm来安装依赖的包
yum -y install gcc make gcc-c++ openssl-devel wget
wget http://nodejs.org/dist/v0.10.26/node-v0.10.26.tar.gz
tar -zvxf node-v0.10.26.tar.gz
cd node-v0.10.26
make && make install
验证是否安装配置成功：
node -v
安装grunt
grunt是一个很方便的构建工具，可以进行打包压缩、测试、执行等等的工作，5.6里的head插件就是通过grunt启动的。因此需要安装一下grunt：
cd /usr/local/elsearch-head
npm install -g grunt-cli //执行后会生成node_modules文件夹
npm install
修改vim Gruntfile.js文件：增加hostname属性，设置为*
connect: {
server: {
options: {
port: 9100,
hostname: '*',
base: '.',
keepalive: true
}
}
}
修改 vim _site/app.js 文件：修改head的连接地址:
this.base_uri = this.config.base_uri || this.prefs.get(“app-base_uri”) || “http://192.168.0.153:9200“;
启动
cd elsearch-head
grunt server & #后台启动
安装go-mysql-elasticsearch进行同步
yum install go
go get github.com/tools/godep
go get github.com/siddontang/go-mysql-elasticsearch
cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch
make
修改配置文件
vim /root/go/src/github.com/siddontang/go-mysql-elasticsearch/etc/river.toml
# MySQL address, user and password
# user must have replication privilege in MySQL.
my_addr = "172.21.0.15:3308"#mysql地址
my_user = "root"#用户名
my_pass = "123456"#密码
# Elasticsearch address
es_addr = "172.21.0.15:9200"#es安装地址
# Path to store data, like master.info, and dump MySQL data
data_dir = "./var"
# Inner Http status address
stat_addr = "172.21.0.15:12800"
# pseudo server id like a slave
server_id = 1
# mysql or mariadb
flavor = "mysql"#mysql数据库
# mysqldump execution path
# if not set or empty, ignore mysqldump.
mysqldump = "mysqldump"
# MySQL data source
[[source]]
schema = "test"#数据库名称
# Only below tables will be synced into Elasticsearch.
# "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
# I don't think it is necessary to sync all tables in a database.
tables = ["travelrecord10","travelrecord4","travelrecord5","travelrecord1"]#数据表名称，可以多个
# Below is for special rule mapping
#[[rule]]
#schema = "test"
#table = "test_river"
#index = "river"
#type = "river"
# only MySQL field in filter will be synced
#filter=["title","tags"]
# title is MySQL test_river field name, es_title is the customized name in Elasticsearch
#[rule.field]
# This will map column title to elastic search my_title
#title="es_title"
# This will map column tags to elastic search my_tags and use array type
#tags="my_tags,list"
# This will map column keywords to elastic search keywords and use array type
#keywords=",list"
# wildcard table rule, the wildcard table must be in source tables
[[rule]]
schema = "test"#数据库名称
table = "travelrecord10"#数据表名称
index = ""#生成es数据索引名称，对应schema
type = ""#生成es数据类型，对应table
#[[rule]]
#schema = "test"#数据库名称
#table = "travelrecord8"#数据表名称
#index = ""#生成es数据索引名称，对应schema
#type = ""#生成es数据类型，对应table
# title is MySQL test_river field name, es_title is the customized name in Elasticsearch
[[rule.fields]]
mysql = "title"
elastic = "es_title"
运行
cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch
./bin/go-mysql-elasticsearch -config=./etc/river.toml > /home/heron/wwwlogs/go-mysql.log 2>&1 &
测试：
db增加数据，查看页面是否显示。
curl -XPOST -u elastic '10.168.2.10:9200/_xpack/security/user/demo' -d '{ "password" : "123456", "full_name" : " elastic", "email" : “elastic@163.com", "roles" : [ "clicks_admin" ] }'
不稳定现况，会自己挂掉，暂时没找到解决办法，只能挂掉立即重启
