• ERROR [org.apache.hadoop.util.Shell]


    2018-04-11 16:23:26,935 DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, value=[Rate of successful kerberos logins and latency (milliseconds)], valueName=Time)
    2018-04-11 16:23:26,937 DEBUG [org.apache.hadoop.metrics2.lib.MutableMetricsFactory] - field org.apache.hadoop.metrics2.lib.MutableRate org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with annotation @org.apache.hadoop.metrics2.annotation.Metric(about=, sampleName=Ops, always=false, type=DEFAULT, value=[Rate of failed kerberos logins and latency (milliseconds)], valueName=Time)
    2018-04-11 16:23:26,938 DEBUG [org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - UgiMetrics, User and group related metrics
    2018-04-11 16:23:26,987 DEBUG [org.apache.hadoop.security.authentication.util.KerberosName] - Kerberos krb5 configuration not found, setting default realm to empty
    2018-04-11 16:23:26,989 DEBUG [org.apache.hadoop.security.Groups] -  Creating new Groups object
    2018-04-11 16:23:26,992 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - Trying to load the custom-built native-hadoop library...
    2018-04-11 16:23:26,995 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: no hadoop in java.library.path
    2018-04-11 16:23:26,995 DEBUG [org.apache.hadoop.util.NativeCodeLoader] - java.library.path=D:SoftwareJavajdk1.8.0_66in;C:WindowsSunJavain;C:Windowssystem32;C:Windows;D:/Software/Java/jdk1.8.0_66/bin/../jre/bin/server;D:/Software/Java/jdk1.8.0_66/bin/../jre/bin;D:/Software/Java/jdk1.8.0_66/bin/../jre/lib/amd64;D:SoftwareAnaconda;D:SoftwareAnacondaLibrarymingw-w64in;D:SoftwareAnacondaLibraryusrin;D:SoftwareAnacondaLibraryin;D:SoftwareAnacondaScripts;C:Windowssystem32;C:Windows;C:WindowsSystem32Wbem;C:WindowsSystem32WindowsPowerShellv1.0;D:SoftwareMySQL Serverin;D:SoftwareCodeBlocksshareCodeBlocksMinGWin;D:Softwarehadoop2.6.0hadoop-2.6.0in;D:SoftwareCTEXUserDatamiktexin;D:SoftwareCTEXMiKTeXmiktexin;D:SoftwareCTEXCTeXctexin;D:SoftwareCTEXCTeXcctin;D:SoftwareCTEXCTeX	yin;D:SoftwareCTEXGhostscriptgs9.05in;D:SoftwareCTEXGSviewgsview;D:SoftwareCTEXWinEdt;D:SoftwareJavajdk1.8.0_66in;D:SoftwareJavajdk1.8.0_66jrein;C:UsersimoAppDataLocalMicrosoftWindowsApps;HADOOP_HOMEin;D:Software搜狐影音5.2.3.15;D:SoftwareJavaeclipse;;.
    2018-04-11 16:23:26,995 WARN [org.apache.hadoop.util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    2018-04-11 16:23:26,997 DEBUG [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Falling back to shell based
    2018-04-11 16:23:27,000 DEBUG [org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback] - Group mapping impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping
    2018-04-11 16:23:27,166 DEBUG [org.apache.hadoop.security.Groups] - Group mapping impl=org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback; cacheTimeout=300000
    2018-04-11 16:23:27,182 DEBUG [org.apache.hadoop.security.UserGroupInformation] - hadoop login
    2018-04-11 16:23:27,184 DEBUG [org.apache.hadoop.security.UserGroupInformation] - hadoop login commit
    2018-04-11 16:23:27,201 DEBUG [org.apache.hadoop.security.UserGroupInformation] - using local user:NTUserPrincipal: Zimo
    2018-04-11 16:23:27,202 DEBUG [org.apache.hadoop.security.UserGroupInformation] - UGI loginUser:Zimo (auth:SIMPLE)
    2018-04-11 16:23:27,395 DEBUG [org.apache.hadoop.hdfs.BlockReaderLocal] - dfs.client.use.legacy.blockreader.local = false
    2018-04-11 16:23:27,395 DEBUG [org.apache.hadoop.hdfs.BlockReaderLocal] - dfs.client.read.shortcircuit = false
    2018-04-11 16:23:27,395 DEBUG [org.apache.hadoop.hdfs.BlockReaderLocal] - dfs.client.domain.socket.data.traffic = false
    2018-04-11 16:23:27,395 DEBUG [org.apache.hadoop.hdfs.BlockReaderLocal] - dfs.domain.socket.path = 
    2018-04-11 16:23:27,435 DEBUG [org.apache.hadoop.metrics2.impl.MetricsSystemImpl] - StartupProgress, NameNode startup progress
    2018-04-11 16:23:27,455 DEBUG [org.apache.hadoop.io.retry.RetryUtils] - multipleLinearRandomRetry = null
    2018-04-11 16:23:27,497 DEBUG [org.apache.hadoop.ipc.Server] - rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@543788f3
    2018-04-11 16:23:27,911 DEBUG [org.apache.hadoop.hdfs.BlockReaderLocal] - Both short-circuit local reads and UNIX domain socket are disabled.
    2018-04-11 16:23:27,917 ERROR [org.apache.hadoop.util.Shell] - Failed to locate the winutils binary in the hadoop binary path
    java.io.IOException: Could not locate executable D:Softwarehadoop2.6.0hadoop-2.6.0inwinutils.exe in the Hadoop binaries.
        at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:278)
        at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:300)
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:293)
        at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
        at org.apache.hadoop.conf.Configuration.getTrimmedStrings(Configuration.java:1546)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:519)
        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:453)
        at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:136)
        at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2433)
        at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88)
        at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:287)
        at com.hadoop.phoneStatistics.ExcelPhoneStatistics.run(ExcelPhoneStatistics.java:95)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at com.hadoop.phoneStatistics.ExcelPhoneStatistics.main(ExcelPhoneStatistics.java:128)
    2018-04-11 16:23:27,975 DEBUG [org.apache.hadoop.ipc.Client] - The ping interval is 60000 ms.
    2018-04-11 16:23:27,976 DEBUG [org.apache.hadoop.ipc.Client] - Connecting to master/192.168.86.131:8020

    解决方法:

      1.未配置环境变量-->配置环境变量HADOOP_HOME,然后重启电脑。或者代码中设置System.setProperty("hadoop.home.dir", "hadoop安装路径")。

      2.还有一种可能HADOOP_HOME的bin目录下根本没有winutils.exe,下载地址https://github.com/srccodes/hadoop-common-2.2.0-bin然后解压到该目录。

    以上就是博主为大家介绍的这一板块的主要内容,这都是博主自己的学习过程,希望能给大家带来一定的指导作用,有用的还望大家点个支持,如果对你没用也望包涵,有错误烦请指出。如有期待可关注博主以第一时间获取更新哦,谢谢!

  • 相关阅读:
    [学习记录]Flask会话维护
    [学习记录]MarkDown语法
    [学习记录]jinja2模板语法
    [学习记录]flask资源加载
    [学习记录]flask初步
    [常用操作]使用github桌面版上传代码
    [学习记录]简明扼要的Sass
    接口Mock测试
    12 | 从0到1:你的第一个GUI自动化测试
    navicat导出DDL语句
  • 原文地址:https://www.cnblogs.com/a-du/p/11017684.html
Copyright © 2020-2023  润新知