• hadoop---wordcount命令


    [zznu@master file]$ hadoop jar ~/hadoop-2.5.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.2.jar wordcount /inputfile output
    16/04/11 22:31:02 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.222.122:18040
    16/04/11 22:31:03 INFO input.FileInputFormat: Total input paths to process : 2
    16/04/11 22:31:03 INFO mapreduce.JobSubmitter: number of splits:2
    16/04/11 22:31:03 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1460438506725_0002
    16/04/11 22:31:03 INFO impl.YarnClientImpl: Submitted application application_1460438506725_0002
    16/04/11 22:31:04 INFO mapreduce.Job: The url to track the job: http://master:18088/proxy/application_1460438506725_0002/
    16/04/11 22:31:04 INFO mapreduce.Job: Running job: job_1460438506725_0002
    16/04/11 22:31:13 INFO mapreduce.Job: Job job_1460438506725_0002 running in uber mode : false
    16/04/11 22:31:13 INFO mapreduce.Job: map 0% reduce 0%
    16/04/11 22:31:26 INFO mapreduce.Job: map 100% reduce 0%
    16/04/11 22:31:34 INFO mapreduce.Job: map 100% reduce 100%
    16/04/11 22:31:34 INFO mapreduce.Job: Job job_1460438506725_0002 completed successfully
    16/04/11 22:31:34 INFO mapreduce.Job: Counters: 49
    File System Counters
    FILE: Number of bytes read=67
    FILE: Number of bytes written=290851
    FILE: Number of read operations=0
    FILE: Number of large read operations=0
    FILE: Number of write operations=0
    HDFS: Number of bytes read=237
    HDFS: Number of bytes written=25
    HDFS: Number of read operations=9
    HDFS: Number of large read operations=0
    HDFS: Number of write operations=2
    Job Counters
    Launched map tasks=2
    Launched reduce tasks=1
    Data-local map tasks=2
    Total time spent by all maps in occupied slots (ms)=21525
    Total time spent by all reduces in occupied slots (ms)=6185
    Total time spent by all map tasks (ms)=21525
    Total time spent by all reduce tasks (ms)=6185
    Total vcore-seconds taken by all map tasks=21525
    Total vcore-seconds taken by all reduce tasks=6185
    Total megabyte-seconds taken by all map tasks=22041600
    Total megabyte-seconds taken by all reduce tasks=6333440
    Map-Reduce Framework
    Map input records=2
    Map output records=6
    Map output bytes=61
    Map output materialized bytes=73
    Input split bytes=200
    Combine input records=6
    Combine output records=5
    Reduce input groups=3
    Reduce shuffle bytes=73
    Reduce input records=5
    Reduce output records=3
    Spilled Records=10
    Shuffled Maps =2
    Failed Shuffles=0
    Merged Map outputs=2
    GC time elapsed (ms)=267
    CPU time spent (ms)=2860
    Physical memory (bytes) snapshot=515694592
    Virtual memory (bytes) snapshot=2516971520
    Total committed heap usage (bytes)=257171456
    Shuffle Errors
    BAD_ID=0
    CONNECTION=0
    IO_ERROR=0
    WRONG_LENGTH=0
    WRONG_MAP=0
    WRONG_REDUCE=0
    File Input Format Counters
    Bytes Read=37
    File Output Format Counters
    Bytes Written=25
    [zznu@master file]$ hadoop fs -ls output
    Found 2 items
    -rw-r--r-- 1 zznu supergroup 0 2016-04-11 22:31 output/_SUCCESS
    -rw-r--r-- 1 zznu supergroup 25 2016-04-11 22:31 output/part-r-00000
    [zznu@master file]$

  • 相关阅读:
    List转换为Datatable
    C# NPOI导出Excel和EPPlus导出Excel比较
    NPOI导出excel2007
    C#中的事务TransactionScope
    switch,case语句易误区
    消息队列mq总结
    一文搞定十大经典排序算法(Java实现)
    常见数据结构与算法整理总结(下)
    常见数据结构与算法整理总结(上)
    mysql开发常用技巧总结
  • 原文地址:https://www.cnblogs.com/ytup/p/5382391.html
Copyright © 2020-2023  润新知