集群版本：CDH6.2.0
集群Hadoop版本：Hadoop 3.0.0-cdh6.2.0
集群HBase版本：2.1.0-cdh6.2.0
操作系统：macOS 10.15.5
IntelliJ IDEA 版本：IntelliJ IDEA 2019.2.3 (Ultimate Edition)
1 Maven HBase Test Project
1.1 安装Maven
参考链接：https://www.jianshu.com/p/3322d6e64a81
1.2 配置Maven阿里云仓库
1.2.1 配置setting.xml
- 配置本地仓库位置
<!-- 本地仓库位置 --> <localRepository>/Users/mo/Maven/m2/repository</localRepository>
- 添加阿里云仓库
1 <!-- 阿里镜像仓库 --> 2 <mirror> 3 <id>alimaven</id> 4 <mirrorOf>central</mirrorOf> 5 <name>aliyun maven</name> 6 <url>http://maven.aliyun.com/nexus/content/repositories/central/</url> 7 </mirror> 8 9 <mirror> 10 <id>nexus-aliyun</id> 11 <mirrorOf>central</mirrorOf> 12 <name>Nexus aliyun</name> 13 <url>http://maven.aliyun.com/nexus/content/groups/public</url> 14 </mirror> 15 16 <mirror> 17 <id>central</id> 18 <name>Maven Repository Switchboard</name> 19 <url>http://repo1.maven.org/maven2/</url> 20 <mirrorOf>central</mirrorOf> 21 </mirror> 22 23 <mirror> 24 <id>repo2</id> 25 <mirrorOf>central</mirrorOf> 26 <name>Human Readable Name for this Mirror.</name> 27 <url>http://repo2.maven.org/maven2/</url> 28 </mirror> 29 30 <mirror> 31 <id>ibiblio</id> 32 <mirrorOf>central</mirrorOf> 33 <name>Human Readable Name for this Mirror.</name> 34 <url>http://mirrors.ibiblio.org/pub/mirrors/maven2/</url> 35 </mirror> 36 37 <mirror> 38 <id>jboss-public-repository-group</id> 39 <mirrorOf>central</mirrorOf> 40 <name>JBoss Public Repository Group</name> 41 <url>http://repository.jboss.org/nexus/content/groups/public</url> 42 </mirror> 43 44 <mirror> 45 <id>mirrorId</id> 46 <mirrorOf>repositoryId</mirrorOf> 47 <name>Human Readable Name for this Mirror.</name> 48 <url>http://my.repository.com/repo/path</url> 49 </mirror> 50 51 </mirrors>
1.3 创建HBase Project
1.3.1 配置IDEA Maven 路径
1.3.2 配置项目pom.xml
- 官方参考配置链接：https://docs.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_6_maven_repo.html
示例：pom.xml
1 <?xml version="1.0" encoding="UTF-8"?> 2 <project xmlns="http://maven.apache.org/POM/4.0.0" 3 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 4 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 5 <modelVersion>4.0.0</modelVersion> 6 7 <groupId>com.cdhHbase</groupId> 8 <artifactId>cdhHbase</artifactId> 9 <version>1.0-SNAPSHOT</version> 10 11 <repositories> 12 <repository> 13 <id>cloudera</id> 14 <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url> 15 </repository> 16 </repositories> 17 18 <properties> 19 <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> 20 <hadoop.version>3.0.0-cdh6.2.0</hadoop.version> 21 <hbase.version>2.1.0-cdh6.2.0</hbase.version> 22 </properties> 23 24 <dependencies> 25 <dependency> 26 <groupId>org.apache.hadoop</groupId> 27 <artifactId>hadoop-client</artifactId> 28 <version>${hadoop.version}</version> 29 </dependency> 30 31 <dependency> 32 <groupId>org.apache.hbase</groupId> 33 <artifactId>hbase-server</artifactId> 34 <version>${hbase.version}</version> 35 </dependency> 36 37 <dependency> 38 <groupId>org.apache.hbase</groupId> 39 <artifactId>hbase-client</artifactId> 40 <version>${hbase.version}</version> 41 </dependency> 42 </dependencies> 43 44 </project>
1.3.3 拷贝CDH相关的配置文件
- 将Hadoop的core-site.xml以及Hbase的hbase-site.xml复制到项目的src/main/resources目录下（CDH 版的相关配置文件在/etc/{module}下）
- 在项目的src/main/resources目录下创建log4j.properties文件（防止日志报红）
示例：log4j.properties
1 log4j.rootLogger=INFO, stdout 2 log4j.appender.stdout=org.apache.log4j.ConsoleAppender 3 log4j.appender.stdout.layout=org.apache.log4j.PatternLayout 4 log4j.appender.stdout.layout.ConversionPattern=%d %p [%c] - %m%n 5 log4j.appender.logfile=org.apache.log4j.FileAppender 6 log4j.appender.logfile.File=target/spring.log 7 log4j.appender.logfile.layout=org.apache.log4j.PatternLayout 8 log4j.appender.logfile.layout.ConversionPattern=%d %p [%c] - %m%n
- OK, Enjoy it.