用Sqoop实现数据HDFS到mysql到Hive

【Hive】用Sqoop实现数据HDFS到mysql到Hive

大数据协作框架

“大数据协作框架”其实是一个统称，主要是以下四个框架

• 数据转换工具    Sqoop
• 文件收集库框架    Flume
• 任务调度框架    Oozie
• 大数据WEB工具Hue

 

Sqoop作用

将关系数据库中的某张表数据抽取到Hadoop的HDFS文件系统中，底层运行的还是MapReduce。

利用MapReduce加快数据传输速度。

批处理方式进行数据传输。

也可以将HDFS上的文件数据或者Hive表中的数据导出到关系型数据库当中的某张表中。

 

HDFS →RDBMS

1. sqoop export
2. --connect jdbc:mysql://xxx:3306/xxx
3. --username xxx
4. --password xxx
5. --table xxx
6. --export-dir xxx

RDBMS→Hive

1. sqoop import
2. --connect jdbc:mysql://xxx:3306/xxx
3. --username xxx
4. --password xxx
5. --fields-terminated-by " "
6. --table xxx
7. --hive-import
8. --hive-table xxx

Hive→RDBMS

1. sqoop export
2. --connect jdbc:mysql://xxx:3306/xxx
3. --username xxx
4. --password xxx
5. --table xxx
6. --export-dir xxx
7. --input-fields-terminated-by ' '

RDBMS→HDFS

1. sqoop import
2. --connect jdbc:mysql://xxx:3306/xxx
3. --username xxx
4. --password xxx
5. --table xxx
6. --target-dir xxx

规律：

1. 从RDBMS导入到HDFS或者Hive中的都使用import；从HDFS或者Hive导出到RDBMS中的都使用export；以HDFS和Hive为参考，根据数据流向选择关键字。
2. connect、username、password、table四个参数为每一种传输都必须的；其中connect参数格式均为--connect jdbc:mysql://主机名:3306/数据库名(使用mysql数据库)；table是指明mysql中的表名。
3. export-dir参数只有在导出数据到RDBMS中时才会用到，含义为表在hdfs中存放的路径。

区别：

• HDFS →RDBMS：指明table：mysql中的表，需要自行先创建；指明export-dir：HDFS中数据的存储路径
• RDBMS→Hive：指明fields-terminated-by：指定分隔符，分隔符指的是存放在Hive中的数据的分隔符，如果目标将存储在Hive中可理解为编码格式，若目标将存储在RDBMS上，则可理解为解码格式；指明table:mysql中的表名；指明hive-import：导入到hive操作;指明hive-table:hive中的表名。注意：table参数不可以与用户家目录下已存在的目录重名，因为sqoop导数据到hive会先将数据导入到HDFS上，然后再将数据load到hive中，最后把这个目录再删除掉。
• Hive→RDBMS：指明table:mysql中的表名；指明export-dir:hive在hdfs中存储的路径;指明hive-table:hive中的表名。
• RDBMS→HDFS：指明table:mysql里的表名;指明target-dir:hdfs存储数据的目录;

 

Sqoop安装

配置Sqoop1.x

conf目录【sqoop-env-template.sh】

• export HADOOP_COMMON_HOME=Hadoop目录
• export HADOOP_MAPRED_HOME=Hadoop目录
• export HIVE_HOME=Hive目录
• export ZOOCFGDIR=Zookeeper目录

将mysqlJDBC驱动包拷到sqoop的lib目录下

测试sqoop

1. bin/sqoop list-databases
2. --connect jdbc:mysql://主机名：3306
3. --username root
4. --password 123456

查看本地mysql

2. mysql> show databases;
3. +--------------------+
4. | Database |
5. +--------------------+
6. | information_schema |
7. | metastore |
8. | mysql |
9. | test |
10. +--------------------+
11. 4 rows in set (0.00 sec)
13. mysql> use test;
14. Reading table information for completion of table and column names
15. You can turn off this feature to get a quicker startup with -A
17. Database changed
18. mysql> show tables;
19. +----------------+
20. | Tables_in_test |
21. +----------------+
22. | my_user |
23. +----------------+
24. 1 row in set (0.00 sec)
26. mysql> desc my_user;
27. +---------+--------------+------+-----+---------+----------------+
28. | Field | Type | Null | Key | Default | Extra |
29. +---------+--------------+------+-----+---------+----------------+
30. | id | tinyint(4) | NO | PRI | NULL | auto_increment |
31. | account | varchar(255) | YES | | NULL | |
32. | passwd | varchar(255) | YES | | NULL | |
33. +---------+--------------+------+-----+---------+----------------+
34. 3 rows in set (0.00 sec)
37. mysql> select * from my_user;
38. +----+----------+----------+
39. | id | account | passwd |
40. +----+----------+----------+
41. | 1 | admin | admin |
42. | 2 | johnny | 123456 |
43. | 3 | zhangsan | zhangsan |
44. | 4 | lisi | lisi |
45. | 5 | test | test |
46. | 6 | qiqi | qiqi |
47. | 7 | hangzhou | hangzhou |
48. +----+----------+----------+
49. 7 rows in set (0.00 sec)

hive创建相同结构的空表

1. hive (test)> create table h_user(
2. > id int,
3. > account string,
4. > passwd string
5. > )row format delimited fields terminated by ' ';
6. OK
7. Time taken: 0.113 seconds
9. hive (test)> desc h_user;
10. OK
11. col_name data_type comment
12. id int
13. account string
14. passwd string
15. Time taken: 0.228 seconds, Fetched: 3 row(s)

从本地mysql导出数据到Hive里

1. bin/sqoop import
2. --connect jdbc:mysql://cdaisuke:3306/test
3. --username root
4. --password 123456
5. --table my_user
6. --num-mappers 1
7. --delete-target-dir
8. --fields-terminated-by " "
9. --hive-database test
10. --hive-import
11. --hive-table h_user
13. hive (test)> select * from h_user;
14. OK
15. h_user.id h_user.account h_user.passwd
16. 1 admin admin
17. 2 johnny 123456
18. 3 zhangsan zhangsan
19. 4 lisi lisi
20. 5 test test
21. 6 qiqi qiqi
22. 7 hangzhou hangzhou
23. Time taken: 0.061 seconds, Fetched: 7 row(s)

从mysql导入到HDFS里

1. bin/sqoop import
2. --connect jdbc:mysql://cdaisuke:3306/test
3. --username root
4. --password 123456
5. --table my_user
6. --num-mappers 3
7. --target-dir /user/hadoop/
8. --delete-target-dir
9. --fields-terminated-by " "
11. ------------------------------------------------------------
13. [hadoop@cdaisuke sqoop-1.4.5-cdh5.3.6]$ bin/sqoop import
14. > --connect jdbc:mysql://cdaisuke:3306/test
15. > --username root
16. > --password 123456
17. > --table my_user
18. > --num-mappers 3
19. > --target-dir /user/hadoop/
20. > --delete-target-dir
21. > --fields-terminated-by " "
22. 18/08/14 00:02:11 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.6
23. 18/08/14 00:02:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
24. 18/08/14 00:02:12 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
25. 18/08/14 00:02:12 INFO tool.CodeGenTool: Beginning code generation
26. 18/08/14 00:02:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `my_user` AS t LIMIT 1
27. 18/08/14 00:02:13 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `my_user` AS t LIMIT 1
28. 18/08/14 00:02:13 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/modules/hadoop-2.5.0-cdh5.3.6_Hive
29. Note: /tmp/sqoop-hadoop/compile/7c8bdb7cd3df7b2f4b48700704f46f65/my_user.java uses or overrides a deprecated API.
30. Note: Recompile with -Xlint:deprecation for details.
31. 18/08/14 00:02:18 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/7c8bdb7cd3df7b2f4b48700704f46f65/my_user.jar
32. 18/08/14 00:02:19 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
33. 18/08/14 00:02:22 INFO tool.ImportTool: Destination directory /user/hadoop is not present, hence not deleting.
34. 18/08/14 00:02:22 WARN manager.MySQLManager: It looks like you are importing from mysql.
35. 18/08/14 00:02:22 WARN manager.MySQLManager: This transfer can be faster! Use the --direct
36. 18/08/14 00:02:22 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
37. 18/08/14 00:02:22 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
38. 18/08/14 00:02:22 INFO mapreduce.ImportJobBase: Beginning import of my_user
39. 18/08/14 00:02:22 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
40. 18/08/14 00:02:22 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
41. 18/08/14 00:02:23 INFO client.RMProxy: Connecting to ResourceManager at slave01/192.168.79.140:8032
42. 18/08/14 00:02:28 INFO db.DBInputFormat: Using read commited transaction isolation
43. 18/08/14 00:02:28 INFO db.DataDrivenDBInputFormat: BoundingValsQuery: SELECT MIN(`id`), MAX(`id`) FROM `my_user`
44. 18/08/14 00:02:28 INFO mapreduce.JobSubmitter: number of splits:3
45. 18/08/14 00:02:28 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1533652222364_0078
46. 18/08/14 00:02:29 INFO impl.YarnClientImpl: Submitted application application_1533652222364_0078
47. 18/08/14 00:02:29 INFO mapreduce.Job: The url to track the job: http://slave01:8088/proxy/application_1533652222364_0078/
48. 18/08/14 00:02:29 INFO mapreduce.Job: Running job: job_1533652222364_0078
49. 18/08/14 00:02:50 INFO mapreduce.Job: Job job_1533652222364_0078 running in uber mode : false
50. 18/08/14 00:02:50 INFO mapreduce.Job: map 0% reduce 0%
51. 18/08/14 00:03:00 INFO mapreduce.Job: map 33% reduce 0%
52. 18/08/14 00:03:01 INFO mapreduce.Job: map 67% reduce 0%
53. 18/08/14 00:03:02 INFO mapreduce.Job: map 100% reduce 0%
54. 18/08/14 00:03:02 INFO mapreduce.Job: Job job_1533652222364_0078 completed successfully
55. 18/08/14 00:03:02 INFO mapreduce.Job: Counters: 30
56. File System Counters
57. FILE: Number of bytes read=0
58. FILE: Number of bytes written=394707
59. FILE: Number of read operations=0
60. FILE: Number of large read operations=0
61. FILE: Number of write operations=0
62. HDFS: Number of bytes read=295
63. HDFS: Number of bytes written=106
64. HDFS: Number of read operations=12
65. HDFS: Number of large read operations=0
66. HDFS: Number of write operations=6
67. Job Counters
68. Launched map tasks=3
69. Other local map tasks=3
70. Total time spent by all maps in occupied slots (ms)=25213
71. Total time spent by all reduces in occupied slots (ms)=0
72. Total time spent by all map tasks (ms)=25213
73. Total vcore-seconds taken by all map tasks=25213
74. Total megabyte-seconds taken by all map tasks=25818112
75. Map-Reduce Framework
76. Map input records=7
77. Map output records=7
78. Input split bytes=295
79. Spilled Records=0
80. Failed Shuffles=0
81. Merged Map outputs=0
82. GC time elapsed (ms)=352
83. CPU time spent (ms)=3600
84. Physical memory (bytes) snapshot=316162048
85. Virtual memory (bytes) snapshot=2523156480
86. Total committed heap usage (bytes)=77766656
87. File Input Format Counters
88. Bytes Read=0
89. File Output Format Counters
90. Bytes Written=106
91. 18/08/14 00:03:02 INFO mapreduce.ImportJobBase: Transferred 106 bytes in 40.004 seconds (2.6497 bytes/sec)
92. 18/08/14 00:03:02 INFO mapreduce.ImportJobBase: Retrieved 7 records.

设置3个map任务

--num-mappers 3

设置HDFS目标存储目录

--target-dir /user/hadoop/

如果设置目录存在则删除此目录

--delete-target-dir

从Hive导出到mysql

在mysql创建新表

1. create table user_export(
2. id tinyint(4) not null auto_increment,
3. account varchar(255) default null,
4. passwd varchar(255) default null,
5. primary key(id)
6. );

用sqoop导出数据

1. bin/sqoop export
2. --connect jdbc:mysql://cdaisuke:3306/test
3. --username root
4. --password 123456
5. --table user_export
6. --num-mappers 1
7. --fields-terminated-by " "
8. --export-dir /user/hive/warehouse/test.db/h_user
10. ----------------------------------------------------
12. [hadoop@cdaisuke sqoop-1.4.5-cdh5.3.6]$ bin/sqoop export
13. > --connect jdbc:mysql://cdaisuke:3306/test
14. > --username root
15. > --password 123456
16. > --table user_export
17. > --num-mappers 1
18. > --fields-terminated-by " "
19. > --export-dir /user/hive/warehouse/test.db/h_user
20. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
21. 18/08/14 00:16:32 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.6
22. 18/08/14 00:16:32 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
23. 18/08/14 00:16:33 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
24. 18/08/14 00:16:33 INFO tool.CodeGenTool: Beginning code generation
25. 18/08/14 00:16:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_export` AS t LIMIT 1
26. 18/08/14 00:16:34 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_export` AS t LIMIT 1
27. 18/08/14 00:16:34 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/modules/hadoop-2.5.0-cdh5.3.6_Hive
28. Note: /tmp/sqoop-hadoop/compile/6823ffae505b34f7ae8b9881bae4b898/user_export.java uses or overrides a deprecated API.
29. Note: Recompile with -Xlint:deprecation for details.
30. 18/08/14 00:16:39 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/6823ffae505b34f7ae8b9881bae4b898/user_export.jar
31. 18/08/14 00:16:39 INFO mapreduce.ExportJobBase: Beginning export of user_export
32. 18/08/14 00:16:40 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
33. 18/08/14 00:16:40 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
34. 18/08/14 00:16:43 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
35. 18/08/14 00:16:43 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
36. 18/08/14 00:16:43 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
37. 18/08/14 00:16:43 INFO client.RMProxy: Connecting to ResourceManager at slave01/192.168.79.140:8032
38. 18/08/14 00:16:48 INFO input.FileInputFormat: Total input paths to process : 1
39. 18/08/14 00:16:48 INFO input.FileInputFormat: Total input paths to process : 1
40. 18/08/14 00:16:48 INFO mapreduce.JobSubmitter: number of splits:1
41. 18/08/14 00:16:48 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
42. 18/08/14 00:16:49 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1533652222364_0079
43. 18/08/14 00:16:50 INFO impl.YarnClientImpl: Submitted application application_1533652222364_0079
44. 18/08/14 00:16:50 INFO mapreduce.Job: The url to track the job: http://slave01:8088/proxy/application_1533652222364_0079/
45. 18/08/14 00:16:50 INFO mapreduce.Job: Running job: job_1533652222364_0079
46. 18/08/14 00:17:11 INFO mapreduce.Job: Job job_1533652222364_0079 running in uber mode : false
47. 18/08/14 00:17:11 INFO mapreduce.Job: map 0% reduce 0%
48. 18/08/14 00:17:27 INFO mapreduce.Job: map 100% reduce 0%
49. 18/08/14 00:17:27 INFO mapreduce.Job: Job job_1533652222364_0079 completed successfully
50. 18/08/14 00:17:27 INFO mapreduce.Job: Counters: 30
51. File System Counters
52. FILE: Number of bytes read=0
53. FILE: Number of bytes written=131287
54. FILE: Number of read operations=0
55. FILE: Number of large read operations=0
56. FILE: Number of write operations=0
57. HDFS: Number of bytes read=258
58. HDFS: Number of bytes written=0
59. HDFS: Number of read operations=4
60. HDFS: Number of large read operations=0
61. HDFS: Number of write operations=0
62. Job Counters
63. Launched map tasks=1
64. Data-local map tasks=1
65. Total time spent by all maps in occupied slots (ms)=13426
66. Total time spent by all reduces in occupied slots (ms)=0
67. Total time spent by all map tasks (ms)=13426
68. Total vcore-seconds taken by all map tasks=13426
69. Total megabyte-seconds taken by all map tasks=13748224
70. Map-Reduce Framework
71. Map input records=7
72. Map output records=7
73. Input split bytes=149
74. Spilled Records=0
75. Failed Shuffles=0
76. Merged Map outputs=0
77. GC time elapsed (ms)=73
78. CPU time spent (ms)=1230
79. Physical memory (bytes) snapshot=113061888
80. Virtual memory (bytes) snapshot=838946816
81. Total committed heap usage (bytes)=45613056
82. File Input Format Counters
83. Bytes Read=0
84. File Output Format Counters
85. Bytes Written=0
86. 18/08/14 00:17:27 INFO mapreduce.ExportJobBase: Transferred 258 bytes in 44.2695 seconds (5.8279 bytes/sec)
87. 18/08/14 00:17:27 INFO mapreduce.ExportJobBase: Exported 7 records.
89. -----------------------------------------------------------------
91. mysql> select * from user_export;
92. +----+----------+----------+
93. | id | account | passwd |
94. +----+----------+----------+
95. | 1 | admin | admin |
96. | 2 | johnny | 123456 |
97. | 3 | zhangsan | zhangsan |
98. | 4 | lisi | lisi |
99. | 5 | test | test |
100. | 6 | qiqi | qiqi |
101. | 7 | hangzhou | hangzhou |
102. +----+----------+----------+
103. 7 rows in set (0.00 sec)

从HDFS导出到mysql

在mysql创建新表

 

1. create table my_user2(
2. id tinyint(4) not null auto_increment,
3. account varchar(255) default null,
4. passwd varchar(255) default null,
5. primary key (id)
6. );
8. ---------------------------------------------------------
10. [hadoop@cdaisuke sqoop-1.4.5-cdh5.3.6]$ bin/sqoop export
11. > --connect jdbc:mysql://cdaisuke:3306/test
12. > --username root
13. > --password 123456
14. > --table my_user2
15. > --num-mappers 1
16. > --fields-terminated-by " "
17. > --export-dir /user/hadoop
18. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
19. 18/08/14 00:39:51 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5-cdh5.3.6
20. 18/08/14 00:39:51 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21. 18/08/14 00:39:52 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
22. 18/08/14 00:39:52 INFO tool.CodeGenTool: Beginning code generation
23. 18/08/14 00:39:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `my_user2` AS t LIMIT 1
24. 18/08/14 00:39:53 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `my_user2` AS t LIMIT 1
25. 18/08/14 00:39:53 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /opt/modules/hadoop-2.5.0-cdh5.3.6_Hive
26. Note: /tmp/sqoop-hadoop/compile/7222f42cd6507a21fdcef7600bd14a20/my_user2.java uses or overrides a deprecated API.
27. Note: Recompile with -Xlint:deprecation for details.
28. 18/08/14 00:39:59 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/7222f42cd6507a21fdcef7600bd14a20/my_user2.jar
29. 18/08/14 00:39:59 INFO mapreduce.ExportJobBase: Beginning export of my_user2
30. 18/08/14 00:40:00 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
31. 18/08/14 00:40:00 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
32. 18/08/14 00:40:04 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
33. 18/08/14 00:40:04 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
34. 18/08/14 00:40:04 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
35. 18/08/14 00:40:04 INFO client.RMProxy: Connecting to ResourceManager at slave01/192.168.79.140:8032
36. 18/08/14 00:40:09 INFO input.FileInputFormat: Total input paths to process : 3
37. 18/08/14 00:40:09 INFO input.FileInputFormat: Total input paths to process : 3
38. 18/08/14 00:40:09 INFO mapreduce.JobSubmitter: number of splits:1
39. 18/08/14 00:40:09 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
40. 18/08/14 00:40:10 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1533652222364_0084
41. 18/08/14 00:40:11 INFO impl.YarnClientImpl: Submitted application application_1533652222364_0084
42. 18/08/14 00:40:11 INFO mapreduce.Job: The url to track the job: http://slave01:8088/proxy/application_1533652222364_0084/
43. 18/08/14 00:40:11 INFO mapreduce.Job: Running job: job_1533652222364_0084
44. 18/08/14 00:40:30 INFO mapreduce.Job: Job job_1533652222364_0084 running in uber mode : false
45. 18/08/14 00:40:30 INFO mapreduce.Job: map 0% reduce 0%
46. 18/08/14 00:40:46 INFO mapreduce.Job: map 100% reduce 0%
47. 18/08/14 00:40:46 INFO mapreduce.Job: Job job_1533652222364_0084 completed successfully
48. 18/08/14 00:40:46 INFO mapreduce.Job: Counters: 30
49. File System Counters
50. FILE: Number of bytes read=0
51. FILE: Number of bytes written=131229
52. FILE: Number of read operations=0
53. FILE: Number of large read operations=0
54. FILE: Number of write operations=0
55. HDFS: Number of bytes read=365
56. HDFS: Number of bytes written=0
57. HDFS: Number of read operations=10
58. HDFS: Number of large read operations=0
59. HDFS: Number of write operations=0
60. Job Counters
61. Launched map tasks=1
62. Data-local map tasks=1
63. Total time spent by all maps in occupied slots (ms)=13670
64. Total time spent by all reduces in occupied slots (ms)=0
65. Total time spent by all map tasks (ms)=13670
66. Total vcore-seconds taken by all map tasks=13670
67. Total megabyte-seconds taken by all map tasks=13998080
68. Map-Reduce Framework
69. Map input records=7
70. Map output records=7
71. Input split bytes=250
72. Spilled Records=0
73. Failed Shuffles=0
74. Merged Map outputs=0
75. GC time elapsed (ms)=89
76. CPU time spent (ms)=1670
77. Physical memory (bytes) snapshot=115961856
78. Virtual memory (bytes) snapshot=838946816
79. Total committed heap usage (bytes)=45613056
80. File Input Format Counters
81. Bytes Read=0
82. File Output Format Counters
83. Bytes Written=0
84. 18/08/14 00:40:46 INFO mapreduce.ExportJobBase: Transferred 365 bytes in 42.3534 seconds (8.618 bytes/sec)
85. 18/08/14 00:40:46 INFO mapreduce.ExportJobBase: Exported 7 records.
87. ------------------------------------------------------------------------
89. mysql> select * from my_user2;
90. +----+----------+----------+
91. | id | account | passwd |
92. +----+----------+----------+
93. | 1 | admin | admin |
94. | 2 | johnny | 123456 |
95. | 3 | zhangsan | zhangsan |
96. | 4 | lisi | lisi |
97. | 5 | test | test |
98. | 6 | qiqi | qiqi |
99. | 7 | hangzhou | hangzhou |
100. +----+----------+----------+
101. 7 rows in set (0.00 sec)