• Eclipse安装Hadoop插件


    首先说明一下配置的环境

    系统:Ubuntu14.0.4

    IDE:eclipse 4.4.1

    Hadoop:Hadoop 2.2.0

            对于Hadoop的旧版本,可以直接复制 hadoop安装目录/contrib/eclipse-plugin/hadoop-0.20.203.0-eclipse-plugin.jar 到 eclipse安装目录/plugins/ 下(并没有亲自验证)。对于Hadoop2,需要上https://github.com/winghc/hadoop2x-eclipse-plugin下载插件的源码自己进行编译生成jar文件。开始为了方便,直接下载别人编译后的jar文件,复制到eclipse的plugins文件夹下,但是重启eclipse后,无法找到Open PersPective中找到Map/Reduce插件。于是猜测插件的编译需要依赖,进而决定自行编译。

          下载插件的源码,上面有清楚的编译方式

    How to build
    
    $cd src/contrib/eclipse-plugin
    
    $ant jar -Dversion=2.2.0 -Declipse.home=/opt/eclipse -Dhadoop.home=/usr/share/hadoop
    
    final jar will be genrated at directory
    
    $root/build/contrib/eclipse-plugin/hadoop-eclipse-plugin-2.2.0.jar
    

      由上可知,需要指定hadoop和eclipse的home目录,一下是我首次编译的结果

    hadoop@yts-Rev-1-0:~/下载/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin$ ant jar -Dversion=2.2.0 -Declipse.home=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop/hadoop-2.2.0
    Buildfile: /home/hadoop/下载/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml
    
    check-contrib:
    
    init:
         [echo] contrib: eclipse-plugin
    
    init-contrib:
    
    ivy-download:
          [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
          [get] To: /home/hadoop/下载/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar
          [get] Not modified - so not downloaded
    
    ivy-probe-antlib:
    
    ivy-init-antlib:
    CLASSPATH element /home/hadoop/下载/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar is not a JAR.
      [typedef] CLASSPATH element /home/hadoop/下载/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar is not a JAR.
      [typedef] Could not load definitions from resource org/apache/ivy/ant/antlib.xml. It could not be found.
    
    BUILD FAILED
    /home/hadoop/下载/hadoop2x-eclipse-plugin-master/src/contrib/build-contrib.xml:476: You need Apache Ivy 2.0 or later from http://ant.apache.org/
          It could not be loaded from http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
    
    Total time: 1 second
    

      编译失败,此时如果电脑是联网状态,会自动下载ivy-2.1.0.jar文件,把下载下来的文件拷贝到上诉的目录中,重新编译

    hadoop@yts-Rev-1-0:~/下载/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin$ ant jar -Dversion=2.2.0 -Declipse.me=/usr/local/eclipse -Dhadoop.home=/usr/local/hadoop/hadoop-2.2.0
    Buildfile: /home/hadoop/下载/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml
    
    check-contrib:
    
    init:
         [echo] contrib: eclipse-plugin
    
    init-contrib:
    
    ivy-download:
          [get] Getting: http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
          [get] To: /home/hadoop/下载/hadoop2x-eclipse-plugin-master/ivy/ivy-2.1.0.jar
          [get] Not modified - so not downloaded
    
    ivy-probe-antlib:
    
    ivy-init-antlib:
    
    ivy-init:
    [ivy:configure] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
    [ivy:configure] :: loading settings :: file = /home/hadoop/下载/hadoop2x-eclipse-plugin-master/ivy/ivysettings.xml
    
    ivy-resolve-common:
    
    ivy-retrieve-common:
    [ivy:cachepath] DEPRECATED: 'ivy.conf.file' is deprecated, use 'ivy.settings.file' instead
    [ivy:cachepath] :: loading settings :: file = /home/hadoop/下载/hadoop2x-eclipse-plugin-master/ivy/ivysettings.xml
    
    compile:
         [echo] contrib: eclipse-plugin
        [javac] /home/hadoop/下载/hadoop2x-eclipse-plugin-master/src/contrib/eclipse-plugin/build.xml:76: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
        [javac] Compiling 45 source files to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/classes
        [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/Path.class): 警告: 无法找到类型 'LimitedPrivate' 的注释方法 'value()': 找不到org.apache.hadoop.classification.InterfaceAudience的类文件
        [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar(org/apache/hadoop/hdfs/DistributedFileSystem.class): 警告: 无法找到类型 'LimitedPrivate' 的注释方法 'value()'
        [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FileSystem.class): 警告: 无法找到类型 'LimitedPrivate' 的注释方法 'value()'
        [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FileSystem.class): 警告: 无法找到类型 'LimitedPrivate' 的注释方法 'value()'
        [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FileSystem.class): 警告: 无法找到类型 'LimitedPrivate' 的注释方法 'value()'
        [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FSDataInputStream.class): 警告: 无法找到类型 'LimitedPrivate' 的注释方法 'value()'
        [javac] /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar(org/apache/hadoop/fs/FSDataOutputStream.class): 警告: 无法找到类型 'LimitedPrivate' 的注释方法 'value()'
        [javac] 注: 某些输入文件使用或覆盖了已过时的 API。
        [javac] 注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
        [javac] 注: 某些输入文件使用了未经检查或不安全的操作。
        [javac] 注: 有关详细信息, 请使用 -Xlint:unchecked 重新编译。
        [javac] 7 个警告
    
    jar:
        [mkdir] Created dir: /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying 9 files to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-app-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-common-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-core-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-hs-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-hs-plugins-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-jobclient-2.2.0-tests.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-jobclient-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-client-shuffle-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-mapreduce-examples-2.2.0.jar
         [copy] Copying 3 files to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0-tests.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-common-2.2.0-tests.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-common-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-common-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/hadoop-nfs-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-nfs-2.2.0.jar
         [copy] Copying 3 files to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0-tests.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-hdfs-2.2.0-tests.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-hdfs-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-hdfs-nfs-2.2.0.jar
         [copy] Copying 11 files to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-api-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-api-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-applications-distributedshell-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-applications-unmanaged-am-launcher-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-client-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-client-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-common-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-common-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-common-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-common-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-nodemanager-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-resourcemanager-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-tests-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-server-web-proxy-2.2.0.jar
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/hadoop-yarn-site-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-yarn-site-2.2.0.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/protobuf-java-2.5.0.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/log4j-1.2.17.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/log4j-1.2.17.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-cli-1.2.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/commons-cli-1.2.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-configuration-1.6.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/commons-configuration-1.6.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/commons-lang-2.5.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/commons-lang-2.5.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-core-asl-1.8.8.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/jackson-core-asl-1.8.8.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/jackson-mapper-asl-1.8.8.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/jackson-mapper-asl-1.8.8.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/slf4j-log4j12-1.7.5.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/slf4j-api-1.7.5.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/guava-11.0.2.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/guava-11.0.2.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/hadoop-auth-2.2.0.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/hadoop-auth-2.2.0.jar
         [copy] Copying 1 file to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib
         [copy] Copying /usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/netty-3.6.2.Final.jar to /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/lib/netty-3.6.2.Final.jar
          [jar] Building jar: /home/hadoop/下载/hadoop2x-eclipse-plugin-master/build/contrib/eclipse-plugin/hadoop-eclipse-plugin-2.2.0.jar
    
    BUILD SUCCESSFUL
    Total time: 36 seconds


    ivy-resolve-common的位置可能会等待一段时间。之后将生成的hadoop-eclipse-plugin-2.2.0.jar拷贝到eclipse的plugins文件夹下,重启eclipse,看见Hdoop插件已经安装成功


    上文记录自己的安装过程,如果有所遗漏或错误,不吝赐教
  • 相关阅读:
    关于缓存雪崩穿透击穿等一些问题
    MethodHandler笔记
    并发总结(博客转载)
    负载均衡的几种算法Java实现代码
    SpringJdbc插入对象返回主键的值
    【Java基础】01-推荐参考材料
    【Java基础】注解
    【JSON】
    【Kafka】3-配置文件说明
    【Kafka】1-理论知识
  • 原文地址:https://www.cnblogs.com/yts1dx/p/4020263.html
Copyright © 2020-2023  润新知