不多说,直接上干货!
对于在oozie里放置Mysql驱动包的总结
根据网上的参考以及我个人经验安装的使用
(1)放一份到$OOZIE_HOME/libext下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
(2)放一份到$OOZIE_HOME/lib下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
(3)放一份到/usr/share/java/下(一定要重命名,mv mysql-connector-java-5.1.21-bin.jar mysql-connector-java.jar)(这里放到bigdatamaster节点)
(4)放一份到$JAVA_HOME/jre/lib/ext (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点、bigdataslave1节点和bigdataslave2节点)
(5)放一份到$OOZIE_HOME/oozie-server/lib下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
(1)放一份到$OOZIE_HOME/libext下
[hadoop@bigdatamaster libext]$ pwd /home/hadoop/app/oozie-4.1.0-cdh5.5.4/libext [hadoop@bigdatamaster libext]$ ls activation-1.1.jar commons-logging-1.1.jar hadoop-mapreduce-client-shuffle-2.6.0-cdh5.5.4.jar leveldbjni-all-1.8.jar apacheds-i18n-2.0.0-M15.jar commons-math3-3.1.1.jar hadoop-yarn-api-2.6.0-cdh5.5.4.jar log4j-1.2.17.jar apacheds-kerberos-codec-2.0.0-M15.jar commons-net-3.1.jar hadoop-yarn-client-2.6.0-cdh5.5.4.jar mysql-connector-java-5.1.21.jar api-asn1-api-1.0.0-M20.jar curator-client-2.7.1.jar hadoop-yarn-common-2.6.0-cdh5.5.4.jar netty-3.6.2.Final.jar api-util-1.0.0-M20.jar curator-framework-2.7.1.jar hadoop-yarn-server-common-2.6.0-cdh5.5.4.jar netty-all-4.0.23.Final.jar avro-1.7.6-cdh5.5.4.jar curator-recipes-2.7.1.jar htrace-core4-4.0.1-incubating.jar paranamer-2.3.jar aws-java-sdk-core-1.10.6.jar ext-2.2 httpclient-4.2.5.jar protobuf-java-2.5.0.jar aws-java-sdk-kms-1.10.6.jar ext-2.2.zip httpcore-4.2.5.jar servlet-api-2.5.jar aws-java-sdk-s3-1.10.6.jar gson-2.2.4.jar jackson-annotations-2.2.3.jar slf4j-api-1.7.5.jar commons-beanutils-1.7.0.jar guava-11.0.2.jar jackson-core-2.2.3.jar slf4j-log4j12-1.7.5.jar commons-beanutils-core-1.8.0.jar hadoop-annotations-2.6.0-cdh5.5.4.jar jackson-core-asl-1.8.8.jar snappy-java-1.0.4.1.jar commons-cli-1.2.jar hadoop-auth-2.6.0-cdh5.5.4.jar jackson-databind-2.2.3.jar stax-api-1.0-2.jar commons-codec-1.4.jar hadoop-aws-2.6.0-cdh5.5.4.jar jackson-jaxrs-1.8.8.jar xercesImpl-2.10.0.jar commons-collections-3.2.2.jar hadoop-client-2.6.0-cdh5.5.4.jar jackson-mapper-asl-1.8.8.jar xml-apis-1.4.01.jar commons-compress-1.4.1.jar hadoop-common-2.6.0-cdh5.5.4.jar jackson-xc-1.8.8.jar xmlenc-0.52.jar commons-configuration-1.6.jar hadoop-hdfs-2.6.0-cdh5.5.4.jar jaxb-api-2.2.2.jar xz-1.0.jar commons-digester-1.8.jar hadoop-mapreduce-client-app-2.6.0-cdh5.5.4.jar jersey-client-1.9.jar zookeeper-3.4.5-cdh5.5.4.jar commons-httpclient-3.1.jar hadoop-mapreduce-client-common-2.6.0-cdh5.5.4.jar jersey-core-1.9.jar commons-io-2.4.jar hadoop-mapreduce-client-core-2.6.0-cdh5.5.4.jar jetty-util-6.1.26.cloudera.2.jar commons-lang-2.4.jar hadoop-mapreduce-client-jobclient-2.6.0-cdh5.5.4.jar jsr305-3.0.0.jar [hadoop@bigdatamaster libext]$
(2)放一份到$OOZIE_HOME/lib下
[hadoop@bigdatamaster lib]$ pwd /home/hadoop/app/oozie-4.1.0-cdh5.5.4/lib [hadoop@bigdatamaster lib]$ ls activemq-client-5.10.2.jar commons-codec-1.4.jar guava-11.0.2.jar jackson-mapper-asl-1.8.8.jar slf4j-simple-1.7.5.jar apacheds-i18n-2.0.0-M15.jar commons-logging-1.1.jar hadoop-auth-2.6.0-cdh5.5.4.jar jline-2.11.jar xercesImpl-2.10.0.jar apacheds-kerberos-codec-2.0.0-M15.jar curator-client-2.7.1.jar hawtbuf-1.10.jar json-simple-1.1.jar xml-apis-1.4.01.jar api-asn1-api-1.0.0-M20.jar curator-framework-2.7.1.jar httpclient-4.2.5.jar jsr305-1.3.9.jar zookeeper-3.4.5-cdh5.5.4.jar api-util-1.0.0-M20.jar geronimo-j2ee-management_1.1_spec-1.0.1.jar httpcore-4.2.5.jar oozie-client-4.1.0-cdh5.5.4.jar commons-cli-1.2.jar geronimo-jms_1.1_spec-1.1.1.jar jackson-core-asl-1.8.8.jar slf4j-api-1.7.5.jar [hadoop@bigdatamaster lib]$ rz [hadoop@bigdatamaster lib]$ ls activemq-client-5.10.2.jar commons-codec-1.4.jar guava-11.0.2.jar jackson-mapper-asl-1.8.8.jar slf4j-api-1.7.5.jar apacheds-i18n-2.0.0-M15.jar commons-logging-1.1.jar hadoop-auth-2.6.0-cdh5.5.4.jar jline-2.11.jar slf4j-simple-1.7.5.jar apacheds-kerberos-codec-2.0.0-M15.jar curator-client-2.7.1.jar hawtbuf-1.10.jar json-simple-1.1.jar xercesImpl-2.10.0.jar api-asn1-api-1.0.0-M20.jar curator-framework-2.7.1.jar httpclient-4.2.5.jar jsr305-1.3.9.jar xml-apis-1.4.01.jar api-util-1.0.0-M20.jar geronimo-j2ee-management_1.1_spec-1.0.1.jar httpcore-4.2.5.jar mysql-connector-java-5.1.21.jar zookeeper-3.4.5-cdh5.5.4.jar commons-cli-1.2.jar geronimo-jms_1.1_spec-1.1.1.jar jackson-core-asl-1.8.8.jar oozie-client-4.1.0-cdh5.5.4.jar [hadoop@bigdatamaster lib]$
[hadoop@bigdatamaster lib]$ chmod 755 mysql-connector-java-5.1.21.jar [hadoop@bigdatamaster lib]$ ll total 9908 -rwxr-xr-x 1 hadoop hadoop 1178428 Apr 26 2016 activemq-client-5.10.2.jar -rwxr-xr-x 1 hadoop hadoop 44925 Apr 26 2016 apacheds-i18n-2.0.0-M15.jar -rwxr-xr-x 1 hadoop hadoop 691479 Apr 26 2016 apacheds-kerberos-codec-2.0.0-M15.jar -rwxr-xr-x 1 hadoop hadoop 16560 Apr 26 2016 api-asn1-api-1.0.0-M20.jar -rwxr-xr-x 1 hadoop hadoop 79912 Apr 26 2016 api-util-1.0.0-M20.jar -rwxr-xr-x 1 hadoop hadoop 41123 Apr 26 2016 commons-cli-1.2.jar -rwxr-xr-x 1 hadoop hadoop 58160 Apr 26 2016 commons-codec-1.4.jar -rwxr-xr-x 1 hadoop hadoop 52915 Apr 26 2016 commons-logging-1.1.jar -rwxr-xr-x 1 hadoop hadoop 69500 Apr 26 2016 curator-client-2.7.1.jar -rwxr-xr-x 1 hadoop hadoop 186273 Apr 26 2016 curator-framework-2.7.1.jar -rwxr-xr-x 1 hadoop hadoop 20220 Apr 26 2016 geronimo-j2ee-management_1.1_spec-1.0.1.jar -rwxr-xr-x 1 hadoop hadoop 32359 Apr 26 2016 geronimo-jms_1.1_spec-1.1.1.jar -rwxr-xr-x 1 hadoop hadoop 1648200 Apr 26 2016 guava-11.0.2.jar -rwxr-xr-x 1 hadoop hadoop 73940 Apr 26 2016 hadoop-auth-2.6.0-cdh5.5.4.jar -rwxr-xr-x 1 hadoop hadoop 50150 Apr 26 2016 hawtbuf-1.10.jar -rwxr-xr-x 1 hadoop hadoop 433368 Apr 26 2016 httpclient-4.2.5.jar -rwxr-xr-x 1 hadoop hadoop 227708 Apr 26 2016 httpcore-4.2.5.jar -rwxr-xr-x 1 hadoop hadoop 227500 Apr 26 2016 jackson-core-asl-1.8.8.jar -rwxr-xr-x 1 hadoop hadoop 668564 Apr 26 2016 jackson-mapper-asl-1.8.8.jar -rwxr-xr-x 1 hadoop hadoop 208781 Apr 26 2016 jline-2.11.jar -rwxr-xr-x 1 hadoop hadoop 16046 Apr 26 2016 json-simple-1.1.jar -rwxr-xr-x 1 hadoop hadoop 33015 Apr 26 2016 jsr305-1.3.9.jar -rwxr-xr-x 1 hadoop hadoop 827942 Nov 3 2016 mysql-connector-java-5.1.21.jar -rwxr-xr-x 1 hadoop hadoop 229725 Apr 26 2016 oozie-client-4.1.0-cdh5.5.4.jar -rwxr-xr-x 1 hadoop hadoop 26084 Apr 26 2016 slf4j-api-1.7.5.jar -rwxr-xr-x 1 hadoop hadoop 10680 Apr 26 2016 slf4j-simple-1.7.5.jar -rwxr-xr-x 1 hadoop hadoop 1363159 Apr 26 2016 xercesImpl-2.10.0.jar -rwxr-xr-x 1 hadoop hadoop 220536 Apr 26 2016 xml-apis-1.4.01.jar -rwxr-xr-x 1 hadoop hadoop 1352025 Apr 26 2016 zookeeper-3.4.5-cdh5.5.4.jar [hadoop@bigdatamaster lib]$
(3)
这个报错很常见,原以为把mysql-connector-java-*-bin.jar丢到oozie的lib即可,但是发现依然宝盖错误。后来在网上找到需要将这个包放到这个路径下就通过了(名字需要修改下)
/usr/share/java/mysql-connector-java.jar
[hadoop@bigdatamaster java]$ pwd /usr/share/java [hadoop@bigdatamaster java]$ su root Password: [root@bigdatamaster java]# ls ant dom3-xerces-j2-2.7.1.jar java_cup-runtime-0.10k.jar js-1.7.jar rhino.jar xml-commons-apis.jar ant-1.7.1.jar dom3-xerces-j2.jar java_cup-runtime.jar js.jar sinjdoc.jar xml-commons-jaxp-1.3-apis.jar ant-bootstrap-1.7.1.jar dom3-xml-commons-apis-1.3.04.jar jaxp13.jar libgcj-4.4.4.jar xerces-j2-2.7.1.jar xml-commons-resolver-1.1.jar ant-bootstrap.jar dom3-xml-commons-apis.jar jaxp.jar libgcj-4.4.7.jar xerces-j2.jar xml-commons-resolver.jar ant.jar gcj-endorsed jaxp_parser_impl.jar rhino-1.7.jar xml-commons-apis-1.3.04.jar ant-launcher-1.7.1.jar java_cup-0.10k.jar jline-0.9.94.jar rhino-examples-1.7.jar xml-commons-apis-ext-1.3.04.jar ant-launcher.jar java_cup.jar jline.jar rhino-examples.jar xml-commons-apis-ext.jar [root@bigdatamaster java]# rz [root@bigdatamaster java]# ls ant dom3-xerces-j2-2.7.1.jar java_cup-runtime-0.10k.jar js-1.7.jar rhino-examples.jar xml-commons-apis-ext.jar ant-1.7.1.jar dom3-xerces-j2.jar java_cup-runtime.jar js.jar rhino.jar xml-commons-apis.jar ant-bootstrap-1.7.1.jar dom3-xml-commons-apis-1.3.04.jar jaxp13.jar libgcj-4.4.4.jar sinjdoc.jar xml-commons-jaxp-1.3-apis.jar ant-bootstrap.jar dom3-xml-commons-apis.jar jaxp.jar libgcj-4.4.7.jar xerces-j2-2.7.1.jar xml-commons-resolver-1.1.jar ant.jar gcj-endorsed jaxp_parser_impl.jar mysql-connector-java-5.1.21.jar xerces-j2.jar xml-commons-resolver.jar ant-launcher-1.7.1.jar java_cup-0.10k.jar jline-0.9.94.jar rhino-1.7.jar xml-commons-apis-1.3.04.jar ant-launcher.jar java_cup.jar jline.jar rhino-examples-1.7.jar xml-commons-apis-ext-1.3.04.jar [root@bigdatamaster java]# ln -s mysql-connector-java-5.1.21.jar mysql-connector-java.jar [root@bigdatamaster java]# ll total 15036 drwxr-xr-x 2 root root 4096 May 11 2016 ant -rw-r--r-- 1 root root 1322826 May 11 2016 ant-1.7.1.jar -rw-r--r-- 1 root root 18834 May 11 2016 ant-bootstrap-1.7.1.jar lrwxrwxrwx 1 root root 23 May 5 20:38 ant-bootstrap.jar -> ant-bootstrap-1.7.1.jar lrwxrwxrwx 1 root root 13 May 5 20:38 ant.jar -> ant-1.7.1.jar -rw-r--r-- 1 root root 12129 May 11 2016 ant-launcher-1.7.1.jar lrwxrwxrwx 1 root root 22 May 5 20:38 ant-launcher.jar -> ant-launcher-1.7.1.jar lrwxrwxrwx 1 root root 19 May 5 20:36 dom3-xerces-j2-2.7.1.jar -> xerces-j2-2.7.1.jar lrwxrwxrwx 1 root root 24 May 5 20:36 dom3-xerces-j2.jar -> dom3-xerces-j2-2.7.1.jar lrwxrwxrwx 1 root root 27 May 5 20:36 dom3-xml-commons-apis-1.3.04.jar -> xml-commons-apis-1.3.04.jar lrwxrwxrwx 1 root root 32 May 5 20:36 dom3-xml-commons-apis.jar -> dom3-xml-commons-apis-1.3.04.jar drwxr-xr-x 2 root root 4096 Mar 22 00:04 gcj-endorsed -rw-r--r-- 1 root root 73310 Aug 21 2010 java_cup-0.10k.jar lrwxrwxrwx 1 root root 18 May 5 20:36 java_cup.jar -> java_cup-0.10k.jar -rw-r--r-- 1 root root 9053 Aug 21 2010 java_cup-runtime-0.10k.jar lrwxrwxrwx 1 root root 26 May 5 20:36 java_cup-runtime.jar -> java_cup-runtime-0.10k.jar lrwxrwxrwx 1 root root 20 May 5 20:36 jaxp13.jar -> xml-commons-apis.jar lrwxrwxrwx 1 root root 20 May 5 20:36 jaxp.jar -> xml-commons-apis.jar lrwxrwxrwx 1 root root 34 May 5 20:36 jaxp_parser_impl.jar -> /etc/alternatives/jaxp_parser_impl -rw-r--r--. 1 root root 84746 Aug 22 2010 jline-0.9.94.jar lrwxrwxrwx. 1 root root 16 May 4 18:10 jline.jar -> jline-0.9.94.jar lrwxrwxrwx. 1 root root 13 May 4 18:10 js-1.7.jar -> rhino-1.7.jar lrwxrwxrwx. 1 root root 10 May 4 18:10 js.jar -> js-1.7.jar -rw-r--r-- 1 root root 10143547 Mar 22 00:05 libgcj-4.4.4.jar lrwxrwxrwx 1 root root 16 May 5 20:36 libgcj-4.4.7.jar -> libgcj-4.4.4.jar -rw-r--r-- 1 root root 827942 Nov 3 2016 mysql-connector-java-5.1.21.jar lrwxrwxrwx 1 root root 31 May 9 14:30 mysql-connector-java.jar -> mysql-connector-java-5.1.21.jar -rw-r--r--. 1 root root 807405 Aug 22 2010 rhino-1.7.jar -rw-r--r--. 1 root root 18260 Aug 22 2010 rhino-examples-1.7.jar lrwxrwxrwx. 1 root root 22 May 4 18:10 rhino-examples.jar -> rhino-examples-1.7.jar lrwxrwxrwx. 1 root root 13 May 4 18:10 rhino.jar -> rhino-1.7.jar -rw-r--r-- 1 root root 414906 Aug 19 2010 sinjdoc.jar -rw-r--r-- 1 root root 1246482 Sep 30 2014 xerces-j2-2.7.1.jar lrwxrwxrwx 1 root root 19 May 5 20:36 xerces-j2.jar -> xerces-j2-2.7.1.jar -rw-r--r-- 1 root root 230522 Aug 19 2010 xml-commons-apis-1.3.04.jar -rw-r--r-- 1 root root 90958 Aug 19 2010 xml-commons-apis-ext-1.3.04.jar lrwxrwxrwx 1 root root 31 May 5 20:36 xml-commons-apis-ext.jar -> xml-commons-apis-ext-1.3.04.jar lrwxrwxrwx 1 root root 27 May 5 20:36 xml-commons-apis.jar -> xml-commons-apis-1.3.04.jar lrwxrwxrwx 1 root root 20 May 5 20:36 xml-commons-jaxp-1.3-apis.jar -> xml-commons-apis.jar -rw-r--r-- 1 root root 56968 Aug 21 2010 xml-commons-resolver-1.1.jar lrwxrwxrwx 1 root root 28 May 5 20:36 xml-commons-resolver.jar -> xml-commons-resolver-1.1.jar [root@bigdatamaster java]#
[root@bigdatamaster java]# mv mysql-connector-java-5.1.21.jar mysql-connector-java.jar mv: overwrite `mysql-connector-java.jar'? y [root@bigdatamaster java]#
(4)因为,我暂时用的是jdk1.7。 即,在$OOZIE_HOME/libext下的mysql-connector-java-6.0.3.jar 替换成 mysql-connector-java-5.1.21.jar 。
(5)放一份到$OOZIE_HOME/oozie-server/lib下 (是 mysql-connector-java-5.1.21.jar)(这里放到bigdatamaster节点)
[hadoop@bigdatamaster lib]$ pwd /home/hadoop/app/oozie-4.1.0-cdh5.5.4/oozie-server/lib [hadoop@bigdatamaster lib]$ ls annotations-api.jar catalina-ha.jar catalina-tribes.jar el-api.jar jasper.jar servlet-api.jar tomcat-dbcp.jar tomcat-i18n-fr.jar catalina-ant.jar catalina.jar ecj-4.3.1.jar jasper-el.jar jsp-api.jar tomcat-coyote.jar tomcat-i18n-es.jar tomcat-i18n-ja.jar [hadoop@bigdatamaster lib]$ rz [hadoop@bigdatamaster lib]$ ls annotations-api.jar catalina-ha.jar catalina-tribes.jar el-api.jar jasper.jar mysql-connector-java-5.1.21.jar tomcat-coyote.jar tomcat-i18n-es.jar tomcat-i18n-ja.jar catalina-ant.jar catalina.jar ecj-4.3.1.jar jasper-el.jar jsp-api.jar servlet-api.jar tomcat-dbcp.jar tomcat-i18n-fr.jar [hadoop@bigdatamaster lib]$ ll total 6348 -rwxr-xr-x 1 hadoop hadoop 15250 May 8 2015 annotations-api.jar -rwxr-xr-x 1 hadoop hadoop 54624 May 8 2015 catalina-ant.jar -rwxr-xr-x 1 hadoop hadoop 132216 May 8 2015 catalina-ha.jar -rwxr-xr-x 1 hadoop hadoop 1250454 May 8 2015 catalina.jar -rwxr-xr-x 1 hadoop hadoop 239165 May 8 2015 catalina-tribes.jar -rwxr-xr-x 1 hadoop hadoop 1830791 May 8 2015 ecj-4.3.1.jar -rwxr-xr-x 1 hadoop hadoop 34622 May 8 2015 el-api.jar -rwxr-xr-x 1 hadoop hadoop 112521 May 8 2015 jasper-el.jar -rwxr-xr-x 1 hadoop hadoop 528303 May 8 2015 jasper.jar -rwxr-xr-x 1 hadoop hadoop 76649 May 8 2015 jsp-api.jar -rw-r--r-- 1 hadoop hadoop 827942 Nov 3 2016 mysql-connector-java-5.1.21.jar -rwxr-xr-x 1 hadoop hadoop 132240 May 8 2015 servlet-api.jar -rwxr-xr-x 1 hadoop hadoop 801374 May 8 2015 tomcat-coyote.jar -rwxr-xr-x 1 hadoop hadoop 253635 May 8 2015 tomcat-dbcp.jar -rwxr-xr-x 1 hadoop hadoop 70563 May 8 2015 tomcat-i18n-es.jar -rwxr-xr-x 1 hadoop hadoop 51715 May 8 2015 tomcat-i18n-fr.jar -rwxr-xr-x 1 hadoop hadoop 54334 May 8 2015 tomcat-i18n-ja.jar [hadoop@bigdatamaster lib]$ chmod 755 mysql-connector-java-5.1.21.jar [hadoop@bigdatamaster lib]$ ll total 6348 -rwxr-xr-x 1 hadoop hadoop 15250 May 8 2015 annotations-api.jar -rwxr-xr-x 1 hadoop hadoop 54624 May 8 2015 catalina-ant.jar -rwxr-xr-x 1 hadoop hadoop 132216 May 8 2015 catalina-ha.jar -rwxr-xr-x 1 hadoop hadoop 1250454 May 8 2015 catalina.jar -rwxr-xr-x 1 hadoop hadoop 239165 May 8 2015 catalina-tribes.jar -rwxr-xr-x 1 hadoop hadoop 1830791 May 8 2015 ecj-4.3.1.jar -rwxr-xr-x 1 hadoop hadoop 34622 May 8 2015 el-api.jar -rwxr-xr-x 1 hadoop hadoop 112521 May 8 2015 jasper-el.jar -rwxr-xr-x 1 hadoop hadoop 528303 May 8 2015 jasper.jar -rwxr-xr-x 1 hadoop hadoop 76649 May 8 2015 jsp-api.jar -rwxr-xr-x 1 hadoop hadoop 827942 Nov 3 2016 mysql-connector-java-5.1.21.jar -rwxr-xr-x 1 hadoop hadoop 132240 May 8 2015 servlet-api.jar -rwxr-xr-x 1 hadoop hadoop 801374 May 8 2015 tomcat-coyote.jar -rwxr-xr-x 1 hadoop hadoop 253635 May 8 2015 tomcat-dbcp.jar -rwxr-xr-x 1 hadoop hadoop 70563 May 8 2015 tomcat-i18n-es.jar -rwxr-xr-x 1 hadoop hadoop 51715 May 8 2015 tomcat-i18n-fr.jar -rwxr-xr-x 1 hadoop hadoop 54334 May 8 2015 tomcat-i18n-ja.jar [hadoop@bigdatamaster lib]$
然后,再,
[hadoop@bigdatamaster oozie-4.1.0-cdh5.5.4]$ bin/oozie-start.sh [hadoop@bigdatamaster oozie-4.1.0-cdh5.5.4]$ jps 10193 Bootstrap 8920 NameNode 9077 SecondaryNameNode 10668 Jps 2182 QuorumPeerMain 9270 ResourceManager [hadoop@bigdatamaster oozie-4.1.0-cdh5.5.4]$
然后呢,大家也许还会出现如下问题: