• spark集群启动命令和Web端口查看


    itcast01上:
    启动hadoop集群:
    cd /itcast/hadoop-2.6.0/sbin/
    ./start-dfs.sh
    启动yarn:
    ./start-yarn.sh
    itcast02上:
    启动resourcemanager
    /itcast/hadoop-2.6.0/sbin/yarn-daemon.sh start resourcemanager
    itcast01上:
    启动spark集群:
    cd /itcast/spark-1.3.1-bin-hadoop2.6/sbin/
    ./start-all.sh
    itcast02上启动spark备用 master节点
    /itcast/spark-1.3.1-bin-hadoop2.6/sbin/start-master.sh
    itcast01上:启动spark shell
    /itcast/spark-1.3.1-bin-hadoop2.6/sbin/start-master.sh
    bin/spark-shell –master

    如果配置了SparkHA,比如zookeeper,在使用spark-submit提交任务时,则需要指定两个master 的信息:spark://itcast01:7077,itcast02:7077

    namenode的webUI端口:50070
    yarn的web端口:8088
    spark集群的web端口:8080
    spark-job监控端口:4040

  • 相关阅读:
    053587
    053586
    053585
    053584
    053583
    053582
    053581
    053580
    053579
    053578
  • 原文地址:https://www.cnblogs.com/shiguangmanbu2016/p/5932819.html
Copyright © 2020-2023  润新知