ElasticSearch学习笔记——插件开发

参考

https://dzone.com/articles/elasticsearch5-how-to-build-a-plugin-and-add-a-lis
https://github.com/chrisshayan/es-changes-feed-plugin
https://blog.csdn.net/qq_16164711/article/details/87872383
https://blog.gaiaproject.club/es-develop-plugin/
https://blog.51cto.com/13755625/2117995

Es的插件主要有如下几种类型，参考

https://github.com/elastic/elasticsearch/tree/master/server/src/main/java/org/elasticsearch/plugins

API Extension Plugins API拓展插件：API extension plugins add new functionality to Elasticsearch by adding new APIs or features, usually to do with search or mapping.

Analysis Plugins 解析器插件：Analysis plugins extend Elasticsearch by adding new analyzers, tokenizers, token filters, or character filters to Elasticsearch.

Alerting Plugins 告警插件：Alerting plugins allow Elasticsearch to monitor indices and to trigger alerts when thresholds are breached.

Discovery Plugins 发现插件：Discovery plugins extend Elasticsearch by adding new discovery mechanisms that can be used instead of Zen Discovery.

Ingest Plugins 摄取插件：The ingest plugins extend Elasticsearch by providing additional ingest node capabilities.

Management Plugins 管理插件：Management plugins offer UIs for managing and interacting with Elasticsearch.

Mapper Plugins Mapper插件：Mapper plugins allow new field datatypes to be added to Elasticsearch.

Security Plugins 安全插件：Security plugins add a security layer to Elasticsearch.

Snapshot/Restore Repository Plugins 快照插件：Repository plugins extend the Snapshot/Restore functionality in Elasticsearch by adding repositories backed by the cloud or by distributed file systems: 

Store Plugins 存储插件：Store plugins offer alternatives to default Lucene stores.

ActionPlugin：An additional extension point for {@link Plugin}s that extends Elasticsearch's scripting functionality

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ActionPlugin.java

CircuitBreakerPlugin 断路器插件：An extension point for {@link Plugin} implementations to add custom circuit breakers

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/CircuitBreakerPlugin.java

ClusterPlugin 集群插件：An extension point for {@link Plugin} implementations to customer behavior of cluster management.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ClusterPlugin.java

EnginePlugin 引擎插件：A plugin that provides alternative engine implementations.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/EnginePlugin.java

ExtensiblePlugin 拓展插件：This class provides a callback for extensible plugins to be informed of other plugins which extend them.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ExtensiblePlugin.java

IndexStorePlugin 索引存储插件：A plugin that provides alternative directory implementations.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/IndexStorePlugin.java

NetworkPlugin 网络插件：Plugin for extending network and transport related classes

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/NetworkPlugin.java

PersistentTaskPlugin 持续任务插件：Plugin for registering persistent tasks executors.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/PersistentTaskPlugin.java

ReloadablePlugin Reload插件：An extension point for {@link Plugin}s that can be reloaded. There is no clear definition about what reloading a plugin actually means. When a plugin is reloaded it might rebuild any internal members. Plugins usually implement this interface in order to reread the values of {@code SecureSetting}s and then rebuild any dependent internal members.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ReloadablePlugin.java

RestCompatibilityPlugin Rest兼容性插件：An extension point for Compatible API plugin implementation.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/RestCompatibilityPlugin.java

ScriptPlugin 脚本插件：An additional extension point for {@link Plugin}s that extends Elasticsearch's scripting functionality.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/ScriptPlugin.java

SearchPlugin 搜索插件：Plugin for extending search time behavior.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/SearchPlugin.java

SystemIndexPlugin 系统索引插件：Plugin for defining system indices. Extends {@link ActionPlugin} because system indices must be accessed via APIs added by the plugin that owns the system index, rather than standard APIs.

https://github.com/elastic/elasticsearch/blob/master/server/src/main/java/org/elasticsearch/plugins/SystemIndexPlugin.java

编写es插件的时候需要提供2类文件

lintong@master:~/software/apache/elasticsearch-6.2.4/plugins/analysis-ik$ ls
commons-codec-1.9.jar    elasticsearch-analysis-ik-6.2.4.jar  httpcore-4.4.4.jar
commons-logging-1.2.jar  httpclient-4.5.2.jar                 plugin-descriptor.properties

一个是plugin-descriptor.properties文件，里面是插件的一些信息，比如

description=自定义插件
version=1.0-SNAPSHOT
name=my plugin
classname=com.xxxx.xx.MetadataListenerPlugin
java.version=1.8
elasticsearch.version=6.2.4

一个是插件的jar包文件

在es启动的过程中，首先会启动节点node，之后PluginsService会加载这个node目录下的插件，可以看到my plugin这个插件已经成功加载

[2021-04-29T10:14:15,061][INFO ][o.e.n.Node               ] [] initializing ...
[2021-04-29T10:14:15,223][INFO ][o.e.e.NodeEnvironment    ] [gl4ygFd] using [1] data paths, mounts [[/media/xxx/14201D6AD04D90DA (/dev/sdb5)]], net usable_space [421.2gb], net total_space [466gb], types [fuseblk]
[2021-04-29T10:14:15,223][INFO ][o.e.e.NodeEnvironment    ] [gl4ygFd] heap size [990.7mb], compressed ordinary object pointers [true]
[2021-04-29T10:14:15,587][INFO ][o.e.n.Node               ] node name [gl4ygFd] derived from node ID [gl4ygFdaRCKyTcLB6SoYJg]; set [node.name] to override
[2021-04-29T10:14:15,588][INFO ][o.e.n.Node               ] version[6.2.4], pid[1162], build[ccec39f/2018-04-12T20:37:28.497551Z], OS[Linux/4.4.0-165-generic/amd64], JVM[Private Build/OpenJDK 64-Bit Server VM/1.8.0_222/25.222-b10]
[2021-04-29T10:14:15,590][INFO ][o.e.n.Node               ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.P9YagNAJ, -XX:+HeapDumpOnOutOfMemoryError, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/lintong/software/apache/elasticsearch-6.2.4, -Des.path.conf=/home/lintong/software/apache/elasticsearch-6.2.4/config]
[2021-04-29T10:14:16,141][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [aggs-matrix-stats]
[2021-04-29T10:14:16,141][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [analysis-common]
[2021-04-29T10:14:16,141][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [ingest-common]
[2021-04-29T10:14:16,142][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [lang-expression]
[2021-04-29T10:14:16,142][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [lang-mustache]
[2021-04-29T10:14:16,142][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [lang-painless]
[2021-04-29T10:14:16,143][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [mapper-extras]
[2021-04-29T10:14:16,143][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [parent-join]
[2021-04-29T10:14:16,143][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [percolator]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [rank-eval]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [reindex]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [repository-url]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [transport-netty4]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded module [tribe]
[2021-04-29T10:14:16,144][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded plugin [analysis-ik]
[2021-04-29T10:14:16,150][INFO ][o.e.p.PluginsService     ] [gl4ygFd] loaded plugin [my plugin]