【Oozie】安装配置Oozie

安装和配置Oozie

Oozie用于Hadoop的工作流配置；

---

参考链接：

《Install and Configure Apache Oozie Workflow Scheduler for CDH 4.X on RHEL/CentOS 6/5》

http://www.tecmint.com/install-apache-oozie-for-cdh-in-centos/  （亲测可用）

《How to Install Latest MySQL 5.7.9 on RHEL/CentOS 7/6/5 and Fedora 23/22/21》

http://www.tecmint.com/install-latest-mysql-on-rhel-centos-and-fedora/  （亲测可用）

主要内容：

• 步骤1：安装Oozie
  
• 步骤2：配置Oozie
  

---

安装Oozie

wget http://archive.cloudera.com/cdh4/one-click-install/redhat/6/x86_64/cloudera-cdh-4-0.x86_64.rpm
yum --nogpgcheck localinstall cloudera-cdh-4-0.x86_64.rpm
yum install oozie
yum install oozie-client

---

配置Oozie

---

首先安装Mysql，可参考：

《How to Install Latest MySQL 5.7.9 on RHEL/CentOS 7/6/5 and Fedora 23/22/21》

http://www.tecmint.com/install-latest-mysql-on-rhel-centos-and-fedora/ 

注意点：执行 mysql_secure_installation时，应该先关闭Mysql，否则报错；

安装步骤：

wget http://dev.mysql.com/get/mysql57-community-release-el6-7.noarch.rpm
yum localinstall mysql57-community-release-el6-7.noarch.rpm 
yum repolist enabled | grep "mysql.*-community.*"
yum install mysql-community-server
service mysqld start/stop/status
mysql --version

查看临时密码：

grep 'temporary password' /var/log/mysqld.log
service mysqld stop
mysql_secure_installation

Sample Output

Securing the MySQL server deployment.
Enter password for user root: Enter New Root Password
VALIDATE PASSWORD PLUGIN can be used to test passwords
and improve security. It checks the strength of password
and allows the users to set only those passwords which are
secure enough. Would you like to setup VALIDATE PASSWORD plugin?
Press y|Y for Yes, any other key for No: y
There are three levels of password validation policy:
LOW    Length >= 8
MEDIUM Length >= 8, numeric, mixed case, and special characters
STRONG Length >= 8, numeric, mixed case, special characters and dictionary                  file
Please enter 0 = LOW, 1 = MEDIUM and 2 = STRONG: 2
Using existing password for root.
Estimated strength of the password: 50 
Change the password for root ? ((Press y|Y for Yes, any other key for No) : y
New password: Set New MySQL Password
Re-enter new password: Re-enter New MySQL Password
Estimated strength of the password: 100 
Do you wish to continue with the password provided?(Press y|Y for Yes, any other key for No) : y
By default, a MySQL installation has an anonymous user,
allowing anyone to log into MySQL without having to have
a user account created for them. This is intended only for
testing, and to make the installation go a bit smoother.
You should remove them before moving into a production
environment.
Remove anonymous users? (Press y|Y for Yes, any other key for No) : y
Success.
Normally, root should only be allowed to connect from
'localhost'. This ensures that someone cannot guess at
the root password from the network.
Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y
Success.
By default, MySQL comes with a database named 'test' that
anyone can access. This is also intended only for testing,
and should be removed before moving into a production
environment.
Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y
- Dropping test database...
Success.
- Removing privileges on test database...
Success.
Reloading the privilege tables will ensure that all changes
made so far will take effect immediately.
Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y
Success.
All done! 

测试：

mysql -u root -p

---

配置Mysql相关属性：

说明：Mysql5.7 对密码的要求比较高，若是修改密码不成功，可以进行如下设置；

# mysql -uroot -p
mysql> SHOW VARIABLES LIKE 'validate_password%';
+--------------------------------------+--------+
| Variable_name                        | Value  |
+--------------------------------------+--------+
| validate_password_dictionary_file    |        |
| validate_password_length             | 8      |
| validate_password_mixed_case_count   | 1      |
| validate_password_number_count       | 1      |
| validate_password_policy             | MEDIUM |
| validate_password_special_char_count | 1      |
+--------------------------------------+--------+
mysql> SET GLOBAL  validate_password_policy='LOW';  
mysql> SET GLOBAL  validate_password_length=4; 

关于安全等级更详细的介绍如下

• LOW 政策只测试密码长度。 密码必须至少有8个字符长。 MEDIUM 政策的条件 密
  
• 码必须包含至少1数字字符,1 大写和小写字符,和1特别 (nonalphanumeric)字符。
  
• STRONG 政策的情况 密码子字符串长度为4的或更长时间不能匹配 单词在字典文件中,如果一个人被指定。
  

[root@master ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or g.
Your MySQL connection id is 3
Server version: 5.5.38 MySQL Community Server (GPL) by Remi
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.
mysql> create database oozie;
Query OK, 1 row affected (0.00 sec)
mysql> grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)
mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';
Query OK, 0 rows affected (0.00 sec)
mysql> exit
Bye

修改Oozie配置文件：

vi /etc/oozie/conf/oozie-site.xml

修改如下属性：（共4个）

<property>
<name>oozie.service.JPAService.jdbc.driver</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.url</name>
<value>jdbc:mysql://localhost:3306/oozie</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.username</name>
<value>oozie</value>
</property>
<property>
<name>oozie.service.JPAService.jdbc.password</name>
<value>oozie</value>
</property>

说明：oozie.service.JPAService.jdbc.url 中的 localhost 实际上是“hostname”执行显示的值；

---

下载并安装MySQL JDBC 连接驱动JAR包到Oozie lib

[root@master oozie]# cd /tmp/
[root@master tmp]# wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.31.tar.gz
[root@master tmp]# tar -zxf mysql-connector-java-5.1.31.tar.gz	
[root@master tmp]# cd mysql-connector-java-5.1.31
[root@master mysql-connector-java-5.1.31]# cp mysql-connector-java-5.1.31-bin.jar /var/lib/oozie/

Create oozie database schema 

[root@master ~]# sudo -u oozie /usr/lib/oozie/bin/ooziedb.sh create -run
Sample Output
setting OOZIE_CONFIG=/etc/oozie/conf
setting OOZIE_DATA=/var/lib/oozie
setting OOZIE_LOG=/var/log/oozie
setting OOZIE_CATALINA_HOME=/usr/lib/bigtop-tomcat
setting CATALINA_TMPDIR=/var/lib/oozie
setting CATALINA_PID=/var/run/oozie/oozie.pid
setting CATALINA_BASE=/usr/lib/oozie/oozie-server-0.20
setting CATALINA_OPTS=-Xmx1024m
setting OOZIE_HTTPS_PORT=11443
...
DONE
Oozie DB has been created for Oozie version '3.3.2-cdh4.7.0'
The SQL commands have been written to: /tmp/ooziedb-8250405588513665350.sql

安装 ExtJS lib以便在网页中浏览

[root@master ~]# cd /tmp/
[root@master tmp]# wget http://archive.cloudera.com/gplextras/misc/ext-2.2.zip
[root@master tmp]# unzip ext-2.2.zip
[root@master tmp]# mv ext-2.2 /var/lib/oozie/

开启Oozie Server

[root@master tmp]# service oozie status
not running.
[root@master tmp]# service oozie start
[root@master tmp]# service oozie status
running
[root@master tmp]# oozie admin -oozie http://localhost:11000/oozie -status
System mode: NORMAL

通过web访问：http://IP:11000

来自为知笔记(Wiz)