Sqoop-1.99.7 安装配置

解压sqoop

下载sqoop安装包

下载地址：

1. http://archive.apache.org/dist/sqoop/1.99.7/ (慢)
2. http://mirrors.hust.edu.cn/apache/sqoop/1.99.7/ （快）

解压到指定目录

tar -zxvf sqoop-1.99.7-bin-hadoop200.tar.gz -C /data/soft/

修改文件夹名

mv sqoop-1.99.7-bin-hadoop200 sqoop-1.99.7

配置sqoop

修改sqoop.properties

(1).修改@LOGDIR@和@BASEDIR@的日志路径 (2).修改mapreduce.configuration.directory的目录

[root@bigdata01 conf]# cat sqoop.properties |grep mapreduce.configuration.directory
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/data/soft/hadoop-3.2.0/etc/hadoop/

(3).配置环境变量(hadoop_home及sqoop_home)

export JAVA_HOME=/usr/local/jdk1.8
export HADOOP_HOME=/data/soft/hadoop-3.2.0
export HIVE_HOME=/data/soft/hive-3.12
export SQOOP_HOME=/data/soft/sqoop-1.99.7
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$SQOOP_HOME/bin:$PATH

(4).修改hadoop中core-site.xml内容

 <property>
        <name>hadoop.proxyuser.root.hosts</name>
        <value>*</value>
    </property>
    <property>
        <name>hadoop.proxyuser.root.groups</name>
        <value>*</value>
    </property>

(5).添加mysql驱动 将mysql驱动包放在sqoop安装目录的server/lib目录下。

[root@bigdata01 lib]# ll |grep mysql
-rw-r--r--. 1 root root   992805 May 25 11:43 mysql-connector-java-5.1.41.jar
-rw-r--r--. 1 root root    18709 Jul 20  2016 sqoop-repository-mysql-1.99.7.jar

验证配置

(1). sqoop2-tool upgrade

[root@bigdata01 lib]# sqoop2-tool upgrade
Setting conf dir: /data/soft/sqoop-1.99.7/bin/../conf
Sqoop home directory: /data/soft/sqoop-1.99.7
Sqoop tool executor:
        Version: 1.99.7
        Revision: 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
        Compiled on Tue Jul 19 16:08:27 PDT 2016 by abefine
Running tool: class org.apache.sqoop.tools.tool.UpgradeTool
0    [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread
Tool class org.apache.sqoop.tools.tool.UpgradeTool has finished correctly.

看到...has finished correctly.表示验证通过 (2). sqoop2-tool verify

[root@bigdata01 lib]# sqoop2-tool verify
Setting conf dir: /data/soft/sqoop-1.99.7/bin/../conf
Sqoop home directory: /data/soft/sqoop-1.99.7
Sqoop tool executor:
        Version: 1.99.7
        Revision: 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb
        Compiled on Tue Jul 19 16:08:27 PDT 2016 by abefine
Running tool: class org.apache.sqoop.tools.tool.VerifyTool
0    [main] INFO  org.apache.sqoop.core.SqoopServer  - Initializing Sqoop server.
6    [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread
Verification was successful.
Tool class org.apache.sqoop.tools.tool.VerifyTool has finished correctly.

启动服务端/客户端

• 启动命令 bash sqoop2-server start

• 关闭命令

  sqoop2-server stop
  

• 打开cli客户端命令

  sqoop2-shell
  

  启动sqoop

  [root@bigdata01 lib]# sqoop2-server start
  Setting conf dir: /data/soft/sqoop-1.99.7/bin/../conf
  Sqoop home directory: /data/soft/sqoop-1.99.7
  Starting the Sqoop2 server...
  0    [main] INFO  org.apache.sqoop.core.SqoopServer  - Initializing Sqoop server.
  10   [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread
  Sqoop2 server started.
  

  启动jps查看进程，可看到SqoopJettyServer 使用客户端连接

  [root@bigdata01 lib]# sqoop2-shell 
  Setting conf dir: /data/soft/sqoop-1.99.7/bin/../conf
  Sqoop home directory: /data/soft/sqoop-1.99.7
  Sqoop Shell: Type 'help' or 'h' for help.
  

sqoop:000>

关闭sqoop服务
``` bash
[root@bigdata01 lib]# sqoop2-server stop
Setting conf dir: /data/soft/sqoop-1.99.7/bin/../conf
Sqoop home directory: /data/soft/sqoop-1.99.7
Stopping the Sqoop2 server...
Sqoop2 server stopped.

至此，sqoop2已完成安装配置。