一、MySQL安装
CenOS7使用MariaDB替代了默认的MySQL。MariaDB数据库管理系统是MySQL的一个分支，主要由开源社区维护，采用GPL授权许可。MariaDB完全兼容MySQL，包括API和命令行。
1.下载MySQL：https://dev.mysql.com/downloads/file/?id=476857
2.将下面的文件复制到master的/home/jun目录下
-rwxrw-rw-. 1 jun jun 25106088 Mar 5 10:24 mysql-community-client-5.7.22-1.el7.x86_64.rpm -rwxrw-rw-. 1 jun jun 280800 Mar 5 10:24 mysql-community-common-5.7.22-1.el7.x86_64.rpm -rwxrw-rw-. 1 jun jun 2239868 Mar 5 10:24 mysql-community-libs-5.7.22-1.el7.x86_64.rpm -rwxrw-rw-. 1 jun jun 172992596 Mar 5 10:25 mysql-community-server-5.7.22-1.el7.x86_64.rpm
3.删除原来已经存在的MariaDB文件包
先检查现有的MariaDB安装包
[root@master jun]# rpm -qa | grep mariadb
mariadb-libs-5.5.56-2.el7.x86_64
删除现有的MariaDB包，其中--nodeps表示强制卸载，即不考虑依赖项
[root@master jun]# rpm -e --nodeps mariadb-libs-5.5.56-2.el7.x86_64
4.安装MySQL，其中i表示安装，v表示更多细节信息，h表示显示进度信息
[root@master jun]# rpm -ivh mysql-community-common-5.7.22-1.el7.x86_64.rpm warning: mysql-community-common-5.7.22-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY Preparing... ################################# [100%] Updating / installing... 1:mysql-community-common-5.7.22-1.e################################# [100%] [root@master jun]# rpm -ivh mysql-community-libs-5.7.22-1.el7.x86_64.rpm warning: mysql-community-libs-5.7.22-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY Preparing... ################################# [100%] Updating / installing... 1:mysql-community-libs-5.7.22-1.el7################################# [100%] [root@master jun]# rpm -ivh mysql-community-client-5.7.22-1.el7.x86_64.rpm warning: mysql-community-client-5.7.22-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY Preparing... ################################# [100%] Updating / installing... 1:mysql-community-client-5.7.22-1.e################################# [100%] [root@master jun]# rpm -ivh mysql-community-server-5.7.22-1.el7.x86_64.rpm warning: mysql-community-server-5.7.22-1.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY Preparing... ################################# [100%] Updating / installing... 1:mysql-community-server-5.7.22-1.e################################# [100%]
5.启动MySQL服务
[root@master jun]# service mysqld start
Redirecting to /bin/systemctl start mysqld.service
6.首次登录获取自动生成的临时密码
[root@master log]# sudo grep 'temporary password' /var/log/mysqld.log 2018-07-23T00:38:31.186026Z 1 [Note] A temporary password is generated for root@localhost: 8k=<lWaofuGN
7.使用临时密码登录MySQL客户端
[root@master log]# mysql -u root -p Enter password: Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 3 Server version: 5.7.22 Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
8.修改登录密码
（1）首先修改成一个符合要去的登录密码
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY 'Abcddcdd_dfa'; Query OK, 0 rows affected (0.00 sec)
（2）查看默认的密码强度设置，可以看到默认的密码要求是：长度至少为8，大小写字母至少各一个，数字至少一个，特殊字符至少各一个，密码强度为MEDIUM（1）。
mysql> select @@validate_password_policy; +----------------------------+ | @@validate_password_policy | +----------------------------+ | MEDIUM | +----------------------------+ 1 row in set (0.00 sec) mysql> show variables like 'validate_password%'; +--------------------------------------+--------+ | Variable_name | Value | +--------------------------------------+--------+ | validate_password_check_user_name | OFF | | validate_password_dictionary_file | | | validate_password_length | 8 | | validate_password_mixed_case_count | 1 | | validate_password_number_count | 1 | | validate_password_policy | MEDIUM | | validate_password_special_char_count | 1 | +--------------------------------------+--------+ 7 rows in set (0.00 sec)
（3）修改默认的密码强度设置
mysql> set global validate_password_policy=0; Query OK, 0 rows affected (0.00 sec) mysql> set global validate_password_mixed_case_count=0; Query OK, 0 rows affected (0.00 sec) mysql> set global validate_password_special_char_count=0; Query OK, 0 rows affected (0.00 sec) mysql> show variables like 'validate_password%'; +--------------------------------------+-------+ | Variable_name | Value | +--------------------------------------+-------+ | validate_password_check_user_name | OFF | | validate_password_dictionary_file | | | validate_password_length | 8 | | validate_password_mixed_case_count | 0 | | validate_password_number_count | 1 | | validate_password_policy | LOW | | validate_password_special_char_count | 0 | +--------------------------------------+-------+ 7 rows in set (0.00 sec)
（4）现在可以修改成想要的登录密码了
mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY '12345678'; Query OK, 0 rows affected (0.00 sec)
9.创建新用户
创建一个名称是hadoopjun的MySQL新用户，并且设置密码，为后面的Hive安装做准备
mysql> grant all on *.* to hadoopjun@'%' identified by '12345678'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> grant all on *.* to hadoopjun@'%localhost' identified by '12345678'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> grant all on *.* to hadoopjun@'%master' identified by '12345678'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec)
10.创建数据库
mysql> create database hive_db; Query OK, 1 row affected (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | hive_db | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.10 sec) mysql> use hive_db Database changed mysql> show tables; Empty set (0.00 sec)
二、Hive的安装与应用
1.确保Hadoop正常启动
2.http://apache.fayea.com/hive下载稳定版hive安装包，并将apache-hive-2.3.3-bin.tar.gz复制到/home/jun下并解压。
3.进入apache-hive-2.3.3-bin/conf目录下使用gedit新建hive-site.xml配置文件
这些配置中，hive_db是之前新建好的数据库，Java连接MySQL使用jdbc驱动，hadoopjun是之前新建的MySQL新用户的名称，12345678是之前新建的MySQL新用户的密码。
注意：&useSSL=false必须要并且xml文件里面的‘&’字符需要转义，不能用‘&’字符
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>hive.metastore.local</name> <value>true</value> </property> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://master:3306/hive_db?characterEncoding=UTF-8&useSSL=false</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hadoopjun</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>12345678</value> </property> </configuration>
4.下载JDBC驱动（https://dev.mysql.com/downloads/file/?id=476197）并将复制到Hive的安装目录即/home/jun/apache-hive-2.3.3-bin/lib下
[jun@master mysql-connector-java-5.1.46]$ cp /home/jun/Resources/mysql-connector-java-5.1.46/mysql-connector-java-5.1.46.jar /home/jun/apache-hive-2.3.3-bin/lib/
5.增加Hive环境变量
使用gedit编辑/home/jun/.bash_profile，增加下面的环境变量，然后需要source才可以
#hive export HIVE_HOME=$PWD/apache-hive-2.3.3-bin export PATH=$PATH:$HIVE_HOME/bin
6.启动并验证Hive
（1）确保MySQL服务已经启动
[jun@master mysql-connector-java-5.1.46]$ service mysqld status Redirecting to /bin/systemctl status mysqld.service ● mysqld.service - MySQL Server Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled) Active: active (running) since Mon 2018-07-23 08:38:36 CST; 1h 11min ago Docs: man:mysqld(8) http://dev.mysql.com/doc/refman/en/using-systemd.html Process: 3749 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS) Process: 3662 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS) Main PID: 3751 (mysqld) Tasks: 29 CGroup: /system.slice/mysqld.service └─3751 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
（3）元数据库初始化，进入/home/jun/apache-hive-2.3.3-bin/bin/，执行下面的命令
[jun@master bin]$ ./schematool -dbType mysql -initSchema
（2）执行hive命令，出现hive>表示安装和部署成功
[jun@master apache-hive-2.3.3-bin]$ hive
which: no hbase in (/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/jun/.local/bin:/home/jun/bin:/home/jun/.local/bin:/home/jun/bin:/home/jun/apache-hive-2.3.3-bin/bin/apache-hive-2.3.3-bin/bin:/home/jun/.local/bin:/home/jun/bin:/home/jun/apache-hive-2.3.3-bin/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/jun/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/jun/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in jar:file:/home/jun/apache-hive-2.3.3-bin/lib/hive-common-2.3.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive>
7.Hive的应用
Hive的应用有两种模式，即命令行模式和交互模式。
（1）交互模式
hive> create database test_db; OK Time taken: 4.583 seconds hive> show databases; OK default test_db Time taken: 0.366 seconds, Fetched: 2 row(s)
（2）命令行模式
[jun@master apache-hive-2.3.3-bin]$ hive -e 'create table testDB(id int, name string, age int);' which: no hbase in (/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/jun/.local/bin:/home/jun/bin:/home/jun/.local/bin:/home/jun/bin:/home/jun/apache-hive-2.3.3-bin/bin/apache-hive-2.3.3-bin/bin:/home/jun/.local/bin:/home/jun/bin:/home/jun/apache-hive-2.3.3-bin/bin) SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/jun/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/jun/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/home/jun/apache-hive-2.3.3-bin/lib/hive-common-2.3.3.jar!/hive-log4j2.properties Async: true OK Time taken: 5.122 seconds [jun@master apache-hive-2.3.3-bin]$ hive -e 'show tables;' which: no hbase in (/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/home/jun/hadoop/bin:/home/jun/hadoop/sbin:/usr/java/jdk1.8.0_171//bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/jun/.local/bin:/home/jun/bin:/home/jun/.local/bin:/home/jun/bin:/home/jun/apache-hive-2.3.3-bin/bin/apache-hive-2.3.3-bin/bin:/home/jun/.local/bin:/home/jun/bin:/home/jun/apache-hive-2.3.3-bin/bin) SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/jun/apache-hive-2.3.3-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/jun/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Logging initialized using configuration in jar:file:/home/jun/apache-hive-2.3.3-bin/lib/hive-common-2.3.3.jar!/hive-log4j2.properties Async: true OK testdb Time taken: 4.313 seconds, Fetched: 1 row(s)