1. Hadoop
下载hadoop-0.23.9-src.tar.gz, 解压, 在Linux 下编译,
mvn package -Pdist,native -DskipTests -Dtar -X
其中会遇到各种错误,主要是linux 软件包安装问题
修改core-site.xml :
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://Host1:9000</value> </property> <!-- <property> <name>hadoop.native.lib</name> <value>false</value> </property> --> <property> <name>io.file.buffer.size</name> <value>131072</value> </property> </configuration>
修改hdfs-site.xml:
<configuration> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>/cmdb/namenode</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>/cmdb/datanode</value> </property> <property> <name>dfs.datanode.max.xcievers</name> <value>4096</value> </property> <property> <name>dfs.namenode.hosts</name> <value>Host2, Host3, Host4</value> </property> </configuration>
修改master,设置SecondaryNameNode:
Host5
修改slaves:
Host2 Host3 Host4
修改配置文件 hadoop-env.sh, 开启JMX监控:
export HADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote.port=10101 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false $HADOOP_NAMENODE_OPTS" export HADOOP_DATANODE_OPTS="-Dcom.sun.management.jmxremote.port=10102 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false $HADOOP_DATANODE_OPTS" export HADOOP_SECONDARYNAMENODE_OPTS="-Dcom.sun.management.jmxremote.port=10103 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false $HADOOP_SECONDARYNAMENODE_OPTS"
2.HBase
下载 hbase-0.94.12.tar.gz, 解压, 编译
mvn package -Dmaven.test.skip.exec=true -Dhadoop.profile=2.0 -Dhadoop.version=0.23 -X
修改hbase-site.xml :
<configuration> <property> <name>hbase.rootdir</name> <value>hdfs://Host1:9000/hbase</value> </property> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>Host1,Host2,Host5</span> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property> <property> <name>hbase.regionserver.wal.codec</name> <value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value> </property> <property> <name>phoenix.query.rowKeyOrderSaltedTable</name> <value>true</value> </property> </configuration>
修改regionservers :
Host2 Host3 Host4
修改hbase-env.sh, 开启JMX:
export HBASE_MASTER_OPTS="-Dcom.sun.management.jmxremote.port=11101 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false $HBASE_MASTER_OPTS" export HBASE_REGIONSERVER_OPTS="-Dcom.sun.management.jmxremote.port=11102 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false $HBASE_REGIONSERVER_OPTS" export HBASE_ZOOKEEPER_OPTS="-Dcom.sun.management.jmxremote.port=11103 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false $HBASE_ZOOKEEPER_OPTS" export HBASE_THRIFT_OPTS="-Dcom.sun.management.jmxremote.port=11104 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false $HBASE_THRIFT_OPTS"