• 大数据平台安装文档


    § 环境安装

    操作系统:Linux iZ42yo1ihl8Z 2.6.32-431.23.3.el6.x86_64 #1 SMP Thu Jul 31 17:20:51 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux。

    § 操作步骤

    § 安装node.js并配置

    § 在服务器新建install目录。

    cd ~ //切换到当前用户目录
    mkdir install     
    cd install//切换到install目录
     

    § 下载最新版本的node.js

    node.js中文网下载最新版本的node-v6.2.0-linux-x64.tar.gz,并利用ftp工具将其上传到install目录下。

    § 解压node.js压缩包

    tar xvf node-v6.2.0-linux-x64.tar.gz
     

    § 配置node.js的环境变量

    切换到解压后的node.js文件夹所在工作目录

    cd node-v6.2.0-linux-x64 
     

    查看此时工作目录的绝对路径

    pwd //home/scapp/install/node-v6.2.0-linux-x64
     

    打开环境变量设置目录

    sudo vi /etc/profile
     

    在末行输入如下配置:wq 保存并退出:

    export NODE_HOME=/home/scapp/install/node-v6.2.0-linux-x64 //上面获取的绝对路径
    export PATH=$PATH:$NODE_HOME/bin
    export NODE_PATH=$PATH:$NODE_HOME/lib/node_modules
     

    让配置生效:

    $ sudo -s
    
    $ source /etc/profile
     

    § 全局安装 pm2

    § 1.在线全局安装pm2

    npm install -g pm2
     

    § 2.离线安装pm2(有pm2依赖包直接跳到第2步)

    1 找一台可以在线安装pm2的机器,执行 npm install pm2 -g 安装pm2,执行 npm config get prefix, 看一下这台机器的npm默认全局安装目录,比如默认安装在/home/scapp/install/node-v6.2.0-linux-x64/lib/node_modules/,然后去这个目录下就能看到你刚刚安装的pm2程序;找到pm2 执行tar czvf pm2.tar.gz pm2 命令, 把pm2打包,在这里打包的好处,是因为该目录下面已经包含了pm2依赖的所有node模块

    2 打完包,同样执行 npm config get prefix 看一下这台服务器的npm默认安装目录为/home/scapp/install/node-v6.2.0-linux-x64/lib/node_modules

    3 执行cd /home/scapp/install/node-v6.2.0-linux-x64/lib/node_modules/命令切换到该目录下,利用ftp工具拷贝你的 pm2.tar.gz 包到该目录下,执行tar xvf pm2.tar.gz解压。

    4 下面就是把这个命令加到系统环境中

    cd /usr/bin
    
    ln -s /home/scapp/install/node-v6.2.0-linux-x64/lib/node_modules/pm2/bin/pm2 pm2
     

    § 离线安装mongodb并配置

    切换到install目录

    cd ~
    cd install
     

    利用ftp工具将压缩包拷贝到install目录,并解压

    tar xvf mongodb-linux-x86_64-3.2.8.tgz
     

    将 test数据库文件夹拷贝到MongoDB的安装目录下

    在安装目录下重启MongoDB服务

    bin/mongod --dbpath ./test/
     

    后台运行MongoDB,在MongoDB安装目录下继续执行:

    bin/mongod -dbpath ./test/ -logpath ./test/mongo.log -logappend -fork -port 27017
     

    § Node.js项目初始化

    § 新建node_root目录,并切换到该目录

    cd ~ //切换到当前用户目录
    mkdir node_root
    cd node_root//切换到node_root目录
     

    § 拷贝相关web服务项目到该目录下

    § 1.直接拷贝express服务压缩包解压到当前目录即可
    unzip pcitc-bigdata-express.zip  //解压express服务压缩包
     
    § 2.利用git克隆远端express服务项目
    git clone git@10.248.10.170:bigdata/pcitc-bigdata-express-index.git //大数据分析平台首页
    
    git clone git@10.248.10.170:bigdata/pcitc-bigdata-express-help.git //大数据分析平台帮助中心
    
    git clone git@10.248.10.170:bigdata/pcitc-bigdata-express-react-platform.git //大数据分析平台工作台
                        
    git clone git@10.248.10.170:bigdata/pcitc-bigdata-express-visualization-service.git //可视化工作台
     

    § 利用pm2命令启动相关服务

    cd /home/scapp/node_root/pcitc-bigdata-express-index
    
    pm2 start ./index-server-3001   //启动大数据分析平台首页服务
    
    cd /home/scapp/node_root/pcitc-bigdata-express-help
    
    pm2 start ./help-server-3009   //启动大数据分析平台帮助中心服务
    
    cd /home/scapp/node_root/pcitc-bigdata-express-react-platform
    
    pm2 start ./bin/react-visualization-server-3083   //启动大数据分析平台工作台服务
    
    cd /home/scapp/node_root/pcitc-bigdata-express-visualization-service
    
    pm2 start ./bin/visualization-service-3081   //启动大数据可视化工作台服务
     

    此时通过http://hostName:3001 http://hostName:3009 http://hostName:3083 进行访问。

    § ngnix反向代理

    § 1.在线安装nginx

    yum install nginx
     

    § 修改配置文件

    
    cd /etc/nginx   
    
    chmod 777 nginx.conf   
    
    vi nginx.conf
     

    将以下配置内容替换原有配置内容:

    
    
    #user  nobody;
    worker_processes  1;
    
    #error_log  logs/error.log;
    #error_log  logs/error.log  notice;
    #error_log  logs/error.log  info;
    
    #pid        logs/nginx.pid;
    
    
    events {
        worker_connections  1024;
    }
    
    
    http {
        include       mime.types;
        default_type  application/octet-stream;
    
        #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
        #                  '$status $body_bytes_sent "$http_referer" '
        #                  '"$http_user_agent" "$http_x_forwarded_for"';
    
        #access_log  logs/access.log  main;
    
        sendfile        on;
        #tcp_nopush     on;
    
        #keepalive_timeout  0;
        keepalive_timeout  65;
    
        #gzip  on;
    
        server {
            listen       80;
            server_name  localhost;
    	#root /ui;
    
            #charset koi8-r;
    
            #access_log  logs/host.access.log  main;
    	location / {
               # root   html;
               # index  index.html index.htm;
    	   # rewrite /(.*) /$1 break;
    	   proxy_pass   http://localhost:3001;
    	   # try_files $uri /index.html;
            }
    
            location /visual/ {
               # root   html;
               # index  index.html index.htm;
               rewrite /visual/(.*) /$1 break;
               proxy_pass   http://localhost:3082;
               # try_files $uri /index.html;
               #  try_files $uri  /visual/index.html;
               #  root html;
    	   #  proxy_pass   http://localhost:3082;
            }
            location /platform/ {
                  rewrite /platform/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
               proxy_pass   http://localhost:3083;
    	}
            location ^~/fonts/{
    	    rewrite "^(.*)/fonts(.*)$" $1/platform/fonts$2;
    	 }
    
           location /conter/{
               rewrite /conter/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
               proxy_pass   http://localhost:3083;
             }
    
           location /view/{
               rewrite /view/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
               proxy_pass   http://localhost:3083;
             }
    
    
    
            location /help/ {
               # root   html;
               #index  index.html index.htm;
               rewrite /help/(.*) /$1 break;
               proxy_pass   http://localhost:3009;
               # try_files $uri /index.html;
            }
    
            location /ui/ {
              rewrite /ui/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
    	   proxy_pass   http://localhost:3097;
            }
    
            location /react/ {
               # root   html;
               # index  index.html index.htm;
               rewrite /react/(.*) /$1 break;
               proxy_pass   http://localhost:3098;
               # try_files $uri /index.html;
            }
         #   location /bootstrap/ {
         #
         #       proxy_pass   http://localhost:3008;
         #   }
    
    
            #error_page  404              /404.html;
    
            # redirect server error pages to the static page /50x.html
            #
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
    
            # proxy the PHP scripts to Apache listening on 127.0.0.1:80
            #
            #location ~ .php$ {
            #    proxy_pass   http://127.0.0.1;
            #}
    
            # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
            #
            #location ~ .php$ {
            #    root           html;
            #    fastcgi_pass   127.0.0.1:9000;
            #    fastcgi_index  index.php;
            #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
            #    include        fastcgi_params;
            #}
    
            # deny access to .htaccess files, if Apache's document root
            # concurs with nginx's one
            #
            #location ~ /.ht {
            #    deny  all;
            #}
        }
    
    
        # another virtual host using mix of IP-, name-, and port-based configuration
        #
        #server {
        #    listen       8000;
        #    listen       somename:8080;
        #    server_name  somename  alias  another.alias;
    
        #    location / {
        #        root   html;
        #        index  index.html index.htm;
        #    }
        #}
    
    
        # HTTPS server
        #
        #server {
        #    listen       443 ssl;
        #    server_name  localhost;
    
        #    ssl_certificate      cert.pem;
        #    ssl_certificate_key  cert.key;
    
        #    ssl_session_cache    shared:SSL:1m;
        #    ssl_session_timeout  5m;
    
        #    ssl_ciphers  HIGH:!aNULL:!MD5;
        #    ssl_prefer_server_ciphers  on;
    
        #    location / {
        #        root   html;
        #        index  index.html index.htm;
        #    }
        #}
    
    }
     

    § 启动ngnix服务

    sudo service nginx start
     

    此时可能出现报错信息: nginx: [emerg] socket() [::]:80 failed (97: Address family not supported by protocol) ,只需修改/etc/nginx/conf.d/default.conf即可

    解决办法:

    
    vim /etc/nginx/conf.d/default.conf
    
    listen       80 default_server;
    listen       [::]:80 default_server;
    
    
    改为:
    listen       80;
    #listen       [::]:80 default_server;
     

    即可,此时通过http://hostName http://hostName/help/ http://hostName/platform/ 进行相关访问。

    § 2.离线安装nginx

    由于nginx依赖于pcre和zlib,所以在安装nginx时,需要先安装pcre和zlib(为了方便起见,我把pcre、zlib、nginx的包放到了同一个目录temp下)

    cd ~ //切换到当前用户目录
    mkdir temp
    cd temp//切换到temp目录
     

    利用ftp工具将 pcre-8.39.tar.gzzlib-1.2.8.tar.gznginx-1.10.1.tar.gz上传到temp目录下

    § 安装pcre

    找到 pcre-8.39.tar.gz 压缩包

    tar -zxvf pcre-8.39.tar.gz 
    cd pcre-8.39
    ./configure
    make
    make install
     

    § 安装zlib

    找到 zlib-1.2.8.tar.gz 压缩包

    tar -xvf zlib-1.2.8.tar.gz
    cd zlib-1.2.8
    ./configure
    make
    make install
     

    § 安装nginx

    找到 nginx-1.10.1.tar.gz 压缩包

    tar -zxvf nginx-1.10.1.tar.gz
    cd nginx-1.10.1.tar.gz
    ./configure --with-pcre=../pcre-8.39 --with-zlib=../zlib-1.2.8
    make
    make install
     

    至此nginx安装完毕

    § 编辑配置文件

    cd /usr/local/nginx/sbin 
    sudo vi ../conf/nginx.conf
     

    将以下配置内容替换原有配置内容:

    
    
    #user  nobody;
    worker_processes  1;
    
    #error_log  logs/error.log;
    #error_log  logs/error.log  notice;
    #error_log  logs/error.log  info;
    
    #pid        logs/nginx.pid;
    
    
    events {
        worker_connections  1024;
    }
    
    
    http {
        include       mime.types;
        default_type  application/octet-stream;
    
        #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
        #                  '$status $body_bytes_sent "$http_referer" '
        #                  '"$http_user_agent" "$http_x_forwarded_for"';
    
        #access_log  logs/access.log  main;
    
        sendfile        on;
        #tcp_nopush     on;
    
        #keepalive_timeout  0;
        keepalive_timeout  65;
    
        #gzip  on;
    
        server {
            listen       80;
            server_name  localhost;
    	#root /ui;
    
            #charset koi8-r;
    
            #access_log  logs/host.access.log  main;
    	location / {
               # root   html;
               # index  index.html index.htm;
    	   # rewrite /(.*) /$1 break;
    	   proxy_pass   http://localhost:3001;
    	   # try_files $uri /index.html;
            }
    
            location /visual/ {
               # root   html;
               # index  index.html index.htm;
               rewrite /visual/(.*) /$1 break;
               proxy_pass   http://localhost:3082;
               # try_files $uri /index.html;
               #  try_files $uri  /visual/index.html;
               #  root html;
    	   #  proxy_pass   http://localhost:3082;
            }
            location /platform/ {
                  rewrite /platform/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
               proxy_pass   http://localhost:3083;
    	}
            location ^~/fonts/{
    	    rewrite "^(.*)/fonts(.*)$" $1/platform/fonts$2;
    	 }
    
           location /conter/{
               rewrite /conter/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
               proxy_pass   http://localhost:3083;
             }
    
           location /view/{
               rewrite /view/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
               proxy_pass   http://localhost:3083;
             }
    
    
    
            location /help/ {
               # root   html;
               #index  index.html index.htm;
               rewrite /help/(.*) /$1 break;
               proxy_pass   http://localhost:3009;
               # try_files $uri /index.html;
            }
    
            location /ui/ {
              rewrite /ui/(.*) /$1 break;
              # root   html;
              # index  index.html index.htm;
    	   proxy_pass   http://localhost:3097;
            }
    
            location /react/ {
               # root   html;
               # index  index.html index.htm;
               rewrite /react/(.*) /$1 break;
               proxy_pass   http://localhost:3098;
               # try_files $uri /index.html;
            }
         #   location /bootstrap/ {
         #
         #       proxy_pass   http://localhost:3008;
         #   }
    
    
            #error_page  404              /404.html;
    
            # redirect server error pages to the static page /50x.html
            #
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
    
            # proxy the PHP scripts to Apache listening on 127.0.0.1:80
            #
            #location ~ .php$ {
            #    proxy_pass   http://127.0.0.1;
            #}
    
            # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
            #
            #location ~ .php$ {
            #    root           html;
            #    fastcgi_pass   127.0.0.1:9000;
            #    fastcgi_index  index.php;
            #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
            #    include        fastcgi_params;
            #}
    
            # deny access to .htaccess files, if Apache's document root
            # concurs with nginx's one
            #
            #location ~ /.ht {
            #    deny  all;
            #}
        }
    
    
        # another virtual host using mix of IP-, name-, and port-based configuration
        #
        #server {
        #    listen       8000;
        #    listen       somename:8080;
        #    server_name  somename  alias  another.alias;
    
        #    location / {
        #        root   html;
        #        index  index.html index.htm;
        #    }
        #}
    
    
        # HTTPS server
        #
        #server {
        #    listen       443 ssl;
        #    server_name  localhost;
    
        #    ssl_certificate      cert.pem;
        #    ssl_certificate_key  cert.key;
    
        #    ssl_session_cache    shared:SSL:1m;
        #    ssl_session_timeout  5m;
    
        #    ssl_ciphers  HIGH:!aNULL:!MD5;
        #    ssl_prefer_server_ciphers  on;
    
        #    location / {
        #        root   html;
        #        index  index.html index.htm;
        #    }
        #}
    
    }
     

    保存配置并退出

    § 非平滑重启

    sudo ./nginx -s stop && sudo ./nginx
     

    nginx离线配置完成,此时通过http://hostName http://hostName/help/ http://hostName/platform/ 进行相关访问。

    § 开源产品界面布局修改

    § Spark分析工具

    zeppelin 0.6.2版本,部署在210节点

    zeppelin.png

    修改了

    /opt/zeppelin-0.6.2-bin-netinst/webapps/webapp/components/navbar navbar.html
    /opt/zeppelin-0.6.2-bin-netinst/webapps/webapp/assets/styles/looknfeel default.css

    § Python分析工具

    jupyter

    部署在207节点

    jupyter.png

    修改了 /home/scapp/anaconda3/pkgs/notebook-4.2.3-py35_0/lib/python3.5/site-packages/notebook/static/style style.min.css

    § R分析工具

    rstudio server

    部署在206节点

    说明:没有进行修改

    § Zabbix监控

    部署在节点211

    Zabbix.png

    修改了

    路径:/var/www/html/zabbix/styles default.css

  • 相关阅读:
    pacx & zr(yet)
    FileInputStream RandomAccessFile FileChannel 与 MappedByteBuffer (yet)
    结合自定义注解的 spring 动态注入
    redis事务与管道区别
    jdk动态代理与cglib优势劣势以及jdk动态代理为什么要interface
    maven scope属性值设置含义
    xc (yet)
    单链表 环
    适配器模式,将老接口的数据给新接口用
    移动硬盘无法拷贝大于4G的文件
  • 原文地址:https://www.cnblogs.com/cxf520/p/6484888.html
Copyright © 2020-2023  润新知