6.搭建sqoop

搭建sqoop

搭建sqoop1.解压安装包2.添加环境变量3.添加mysql驱动4.添加配置文件5.查看是否配置成功

1.解压安装包

1. 在/opt目录下新建sqoop文件夹，上传sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

   tar -zxvf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz

2. 进行重命名

   mv sqoop-1.4.7.bin__hadoop-2.6.0 sqoop-1.4.7

2.添加环境变量

1. 在/etc/profile中，添加

   #sqoop
   export SQOOP_HOME=/opt/sqoop/sqoop-1.4.7
   export PATH=.:${JAVA_HOME}/bin:${SCALA_HOME}/bin:${SPARK_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:${ZK_HOME}/bin:${HBASE_HOME}/bin:${HIVE_HOME}/bin:${KAFKA_HOME}/bin:${SQOOP_HOME}/bin:${FLUME_HOME}/bin:${STORM_HOME}/bin:$PATH
   export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib:$SQOOP_HOME/lib

3.添加mysql驱动

1. 将mysql-connector-java-5.1.36.jar上传到/opt/sqoop/sqoop-1.4.7/lib/下

4.添加配置文件

1. 在/opt/sqoop/sqoop-1.4.7/conf⽬录下，添加配置⽂件，从模板复制就好

   cp sqoop-env-template.sh sqoop-env.sh
   vim sqoop-env.sh

2. 添加如下

   #Set path to where bin/hadoop is available
   export HADOOP_COMMON_HOME=/opt/hadoop/hadoop-2.7.3
   ​
   #Set path to where hadoop-*-core.jar is available
   export HADOOP_MAPRED_HOME=/opt/hadoop/hadoop-2.7.3
   ​
   #set the path to where bin/hbase is available
   #export HBASE_HOME=/opt/hbase/hbase-1.2.6
   ​
   #Set the path to where bin/hive is available
   export HIVE_HOME=/opt/hive/apache-hive-2.3.7-bin
   ​
   #Set the path for where zookeper config dir is
   export ZOOCFGDIR=opt/zookeeper/zookeeper-3.4.5
   ​

5.查看是否配置成功

1. 执⾏sqoop -version,看到版本信息，即安装成功

2. 如果出现如下错误信息：

   Error: Could not find or load main class org.apache.sqoop.Sqoop

   把sqoop⽬录下的sqoop-1.4.6.jar拷⻉到 hadoop/share/hadoop/mapreduce/的lib⽬录下解决 重新sqoop -version,错误不⻅，⾄此，sqoop安装完成