最近将Druid-0.10.0升级到Druid-0.12.1的过程中遇到一些问题，为了后期方便分析问题和及时解决问题，特此写这篇文章将工作中遇到的Druid问题及解决办法记录下来，以供其他人借鉴，其中如有不妥之处，还望大家多多见谅！！
1、协调节点无法创建任务
协调节点无法创建任务基本可以从以下两个方面进行考虑：
- 任务信息已在数据库中存在
- 非堆最大内存小于实际所需内存
实时节点报错信息：
Error: com.metamx.tranquility.druid.IndexServicePermanentException: Service[druid:overlord] call failed with status: 400 Bad Request
任务报错信息：
Not enough direct memory. Please adjust -XX:MaxDirectMemorySize, druid.processing.buffer.sizeBytes, druid.processing.numThreads, or druid.processing.numMergeBuffers: maxDirectMemory[1,908,932,608], memoryNeeded[2,684,354,560] = druid.processing.buffer.sizeBytes[536,870,912] * (druid.processing.numMergeBuffers[2] + druid.processing.numThreads[2] + 1)
解决方法：
调小MaxDirectMemorySize或者修改common.runtime.config中的processing.numthreads
-server
-Xms24g
-Xmx24g
-XX:MaxDirectMemorySize=4096m
-Duser.timezone=UTC
-Dfile.encoding=UTF-8
-Djava.io.tmpdir=var/tmp
-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager
注意：这个问题常见于Historical和MiddleManager节点。如果出现此类问题，首先应该排查Historical和MIddleManager节点的MaxDirectMemorySize配置的值。
Historical和MiddleManager节点配置MaxDirectMemorySize必须满足下面这个公式：
MaxDirectMemorySize >= druid.processing.buffer.sizeBytes[536,870,912] * (druid.processing.numMergeBuffers[2] + druid.processing.numThreads[2] + 1)
2、数据进行GroupBy时聚合数量大从而导致报错
出现该错误主要是由以下两方面导致：
- druid.processing.buffer.sizeBytes配置的缓存数据过小
- druid.query.groupBy.maxOnDiskStorage默认关闭，未开启磁盘溢出功能
错误信息：
io.druid.query.QueryInterruptedException: Not enough dictionary space to execute this query. Try increasing druid.query.groupBy.maxMergingDictionarySize or enable disk spilling by setting druid.query.groupBy.maxOnDiskStorage to a positive number.
解决方法：
- Broker、Historical以及实时节点更改druid.processing.buffer.sizeBytes属性值。
druid.processing.buffer.sizeBytes=536870912
- Broker、Historical以及实时节点开启磁盘溢出和配置最大磁盘存储空间，具体配置如下所示：
druid.query.groupBy.maxOnDiskStorage=1
druid.query.groupBy.maxOnDiskStorage=6442450944
注意：这两种方式中可以任选其中一种，修改配置后，只需要重启相应的节点即可。
Druid创建任务时出现端口占用的情况
这种异常情况下，druid能够正常在mysql中添加task记录，但是没法在var/druid/task目录下创建segment，从而导致无法将数据保存到Druid中。从0.12.0版本开始偶尔会出现这种情况。
报错信息：
2018-07-25T05:00:14,262 WARN [main] com.sun.jersey.spi.inject.Errors - The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void io.druid.server.http.SegmentListerResource.getSegments(long,long,long,javax.servlet.http.HttpServletRequest) throws java.io.IOException, MUST return a non-void type.
2018-07-25T05:00:14,273 INFO [main] org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler@915d7c4{/,null,AVAILABLE}
2018-07-25T05:00:14,277 ERROR [main] io.druid.cli.CliPeon - Error when starting up. Failing.
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_65]
at sun.nio.ch.Net.bind(Net.java:433) ~[?:1.8.0_65]
at sun.nio.ch.Net.bind(Net.java:425) ~[?:1.8.0_65]
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) ~[?:1.8.0_65]
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) ~[?:1.8.0_65]
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:317) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.3.19.v20170502.jar:9.3.19.v20170502]
at org.eclipse.jetty.server.Server.doStart(Server.java:401) ~[jetty-server-9.3.19.v20170502.jar:9.3.19.v20170502]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) ~[jetty-util-9.3.19.v20170502.jar:9.3.19.v20170502]
at io.druid.server.initialization.jetty.JettyServerModule$1.start(JettyServerModule.java:315) ~[druid-server-0.12.1.jar:0.12.1]
at io.druid.java.util.common.lifecycle.Lifecycle.start(Lifecycle.java:311) ~[java-util-0.12.1.jar:0.12.1]
at io.druid.guice.LifecycleModule$2.start(LifecycleModule.java:134) ~[druid-api-0.12.1.jar:0.12.1]
at io.druid.cli.GuiceRunnable.initLifecycle(GuiceRunnable.java:101) [druid-services-0.12.1.jar:0.12.1]
at io.druid.cli.CliPeon.run(CliPeon.java:301) [druid-services-0.12.1.jar:0.12.1]
at io.druid.cli.Main.main(Main.java:116) [druid-services-0.12.1.jar:0.12.1]
解决办法：
MiddleManager节点的配置文件runtime.properties添加如下配置：
cat druid.indexer.runner.startPort=40000
注意事项：先查看linux服务器支持的端口范围，然后根据端口范围定义任务执行起始端口
$ sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 32768 60999
3. Segment进行compact时临时文件不存在导致报错
出现该错误主要是由于MiddleManager节点指定的临时文件不存在
MiddleManager节点配置文件jvm.properties中指定临时目录
-Djava.io.tmpdir=var/tmp
错误信息
java.lang.IllegalStateException: Failed to create directory within 10000 attempts (tried 1472453270713-0 to 1472453270713-9999)
解决办法
手动创建临时目录，比如上面的临时目录var/tmp
[work@localhost ~]$ mkdir -p var/tmp