build hadoop, spark, hbase cluster

1,something:

1,arc land

  506  git branch

  507  git status

  508  git reset multicloud/qcloud/cluster_management/spark_cluster/dependent_file/spark-1.6.2-bin-hadoop2.6.tgz

  509  git status

  510  git add multicloud/qcloud/tools/ssh.py

  511  git status

  512  git log

  513  git commit -a

  514  git status

  515  git log

  516  git pull --rebase origin master

  517  git log

  518  arc diff

  519  arc diff

  520  history

git checkout -b test1
  432  git status
  433  git branch
  434  ls
  435  cd core/
  436  ls
  437  cd ..
  438  cd multicloud/
  439  ls
  440  cd qcloud/
  441  ls
  442  cd ..
  443  cd ..
  444  ls
  445  ls multicloud/qcloud/spark_cluster/README.md 
  446  vim multicloud/qcloud/spark_cluster/README.md 
  447  git branch
  448  git status
  449  git commit -a 'test1'
  450  git commit -a
  451  git log
  452  clear
  453  git pull --rebase origin master
  454  arc diff
  455  pwd
  456  arc help
  457  source ~/.bashrc 
  458  arc
  459  arc help
  460  ls ~/.bashrc 
  461  arc diff
  462  arc diff
  463  arc land

2,ssh aliyun

ssh -i aliyun.pem root@101.201.233.28

chmod 600 aliyun.pem 

3,ssh免密码登录

(1)

ssh-keygen -t rsa

scp -i ~/aliyun.pem id_rsa.pub root@172.17.148.92:~/.ssh

(2)in 172.17.148.92

cat id_rsa.pub >> authorized_keys

chmod 600 authorized_keys

http://blog.csdn.net/m15851813962/article/details/53150967

4,docker

docker exec -it ssss /bin/bash

5, scp:

scp -i ~/work/keys/alicloud.pem * hadoopmaster:/root

6, firewall

systemctl stop firewalld

7,auto build command

spark:

python auto_build_spark.py -f /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/dev_conf/alicloud_dev_spark_cluster_conf.json -k /Users/chunyueli/work/keys/alicloud.pem -F /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/restart_qcloud_cluster.py -i 114.240.80.157 -a restart

hadoop

python auto_build_hadoop.py -f /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/dev_conf/alicloud_dev_hadoop_cluster_conf.json -k /Users/chunyueli/work/keys/alicloud.pem -F /Users/chunyueli/work/github/infra/multicloud/qcloud/cluster_management/spark_cluster/restart_qcloud_cluster.py -i 114.240.85.26

8, docker command