• Doris安装


    1、下载安装

    2、解压

    tar -xf apache-doris_apache-doris-0.12.0-incubating-bin.tar.gz -C /usr/java
    
    mv  apache-doris_apache-doris-0.12.0 dorsi

    3、启动doris。 参考博客:http://doc.dorisdb.com/2178813

    1、安装mysql-client失败:Error: Nothing to dothing to do
      a) 添加源 rpm -ivh https://repo.mysql.com//mysql57-community-release-el7-11.noarch.rpm
    b) yum search mysql-community
    c) yum -y install mysql-community-client.x86_64
    FE:
    2、cd DorisDB-XX-1.0.0/fe
    
    3、mkdir -p doris-meta
    
    4、bin/start_fe.sh --daemon
    
    5、查看log日志,8030和9030是否启动成功,且没有错误
    
    6、mysql -h 127.0.0.1 -P9030 -uroot 连接doris
    
    7、SHOW PROC '/frontends'G 查看状态
    
    BE:
    
    8、cd DorisDB-XX-1.0.0/be
    
    9、mkdir -p storage
    
    10、ALTER SYSTEM ADD BACKEND "hadoop102:9050";
    
    11、bin/start_be.sh --daemon
    
    12、SHOW PROC '/backends'G  
    如果isAlive为true,则说明BE正常接入集群。如果BE没有正常接入集群,请查看log目录下的be.WARNING日志文件确定原因。

    ########################案例,导入CSV文件############################# 

    1、创建表结构

    CREATE DATABASE IF NOT EXISTS load_test;
     
     USE load_test;
     
     CREATE TABLE insert_wiki_edit
    (
        event_time DATETIME,
        channel VARCHAR(32) DEFAULT '',
        user VARCHAR(128) DEFAULT '',
        is_anonymous TINYINT DEFAULT '0',
        is_minor TINYINT DEFAULT '0',
        is_new TINYINT DEFAULT '0',
        is_robot TINYINT DEFAULT '0',
        is_unpatrolled TINYINT DEFAULT '0',
        delta INT SUM DEFAULT '0',
        added INT SUM DEFAULT '0',
        deleted INT SUM DEFAULT '0'
    )
    AGGREGATE KEY(event_time, channel, user, is_anonymous, is_minor, is_new, is_robot, is_unpatrolled)
    PARTITION BY RANGE(event_time)
    (
        PARTITION p06 VALUES LESS THAN ('2015-09-12 06:00:00'),
        PARTITION p12 VALUES LESS THAN ('2015-09-12 12:00:00'),
        PARTITION p18 VALUES LESS THAN ('2015-09-12 18:00:00'),
        PARTITION p24 VALUES LESS THAN ('2015-09-13 00:00:00')
    )
    DISTRIBUTED BY HASH(user) BUCKETS 10
    PROPERTIES("replication_num" = "1");

    2、导入数据

    curl --location-trusted -u root -H "column_separator:," -T wikipedia-2015-09-12-sampled http://localhost:8030/api/load_test/insert_wiki_edit/_stream_load

    解释:
    -u root :表示doris的账户,其实可以写成 -u root:password
      -H "column_separator:,"  : 表示csv文件分割的字符串
    -T wikipedia-2015-09-12-sampled   : 表示数据文件
     load_test/insert_wiki_edit/_stream_load   :表示  库名/表名/执行方式

    3、导入报错:

    {
        "TxnId": 7,
        "Label": "2eb7f173-d107-4169-aafd-951a6aced38d",
        "Status": "Fail",
        "Message": "too many filtered rows",
        "NumberTotalRows": 39244,
        "NumberLoadedRows": 39242,
        "NumberFilteredRows": 2,
        "NumberUnselectedRows": 0,
        "LoadBytes": 2505414,
        "LoadTimeMs": 426,
        "ErrorURL": "http://192.168.75.102:8040/api/_load_error_log?file=__shard_4/error_log_insert_stmt_f145238f303b5cc5-a0e5691877cab184_f145238f303b5cc5_a0e5691877cab184"
    }

    可以查看错误curl   http://192.168.75.102:8040/api/_load_error_log?file=__shard_4/error_log_insert_stmt_f145238f303b5cc5-a0e5691877cab184_f145238f303b5cc5_a0e5691877cab184  

    Reason: actual column number is more than schema column number. actual number: 12 sep: ,, schema number: 11; . src line: [2015-09-12 15:00:00,#en.wikipedia,Eat me, I'm a red bean,0,0,0,0,0,381,381,0]; 
    Reason: actual column number is more than schema column number. actual number: 12 sep: ,, schema number: 11; . src line: [2015-09-12 20:00:00,#pt.wikipedia,Chronus, o cagão bunda-mole corno,0,0,0,0,1,0,0,0]; 

    从错误可知,实际由12列,但是schema只有11列

     4、导入成功

    {
        "TxnId": 8,
        "Label": "3b68d252-af25-4a55-b126-1310cd92931d",
        "Status": "Success",
        "Message": "OK",
        "NumberTotalRows": 39244,
        "NumberLoadedRows": 39244,
        "NumberFilteredRows": 0,
        "NumberUnselectedRows": 0,
        "LoadBytes": 2505412,
        "LoadTimeMs": 323
    }

     5、集群信息

    错误:

    mysql> show backends ;
    +-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------------------------------------------------------+---------------------+
    | BackendId | Cluster         | IP             | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime       | LastHeartbeat       | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | ErrMsg                                                             | Version             |
    +-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------------------------------------------------------+---------------------+
    | 10002     | default_cluster | 192.168.75.101 | 9050          | 9060   | 8040     | 8060     | 2021-06-29 14:18:36 | 2021-06-29 14:22:16 | true  | false                | false                 | 10        | 573.000 B        | 30.567 GB     | 75.048 GB     | 59.27 % | 59.27 %        |                                                                    | 0.12.0-rc03-Unknown |
    | 10122     | default_cluster | 192.168.75.102 | 9050          | 9060   | 8040     | 8060     | N/A                 | N/A                 | false | false                | false                 | 0         | .000             | 1.000 B       | .000          | 0.00 %  | 0.00 %         | java.net.ConnectException: Connection refused (Connection refused) |                     |
    | 10123     | default_cluster | 192.168.75.103 | 9050          | 9060   | 8040     | 8060     | N/A                 | N/A                 | false | false                | false                 | 0         | .000             | 1.000 B       | .000          | 0.00 %  | 0.00 %         | java.net.ConnectException: Connection refused (Connection refused) |                     |
    +-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------------------------------------------------------------------+---------------------+
    3 rows in set (0.01 sec)
    
    mysql> show frontend;
    ERROR 1064 (HY000): errCode = 2, detailMessage = Syntax error in line 1:
    show frontend
         ^
    Encountered: FRONTEND
    Expected
    
    mysql> show frontends;
    +----------------------------------+---------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+
    | Name                             | IP            | EditLogPort | HttpPort | QueryPort | RpcPort | Role     | IsMaster | ClusterId  | Join | Alive | ReplayedJournalId | LastHeartbeat       | IsHelper | ErrMsg |
    +----------------------------------+---------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+
    | 192.168.122.1_9010_1624848281806 | 192.168.122.1 | 9010        | 8030     | 9030      | 9020    | FOLLOWER | true     | 2117830509 | true | true  | 29846             | 2021-06-29 14:22:51 | true     |        |
    +----------------------------------+---------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+
    1 row in set (0.03 sec)

    正确:

    mysql> show backends;
    +-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+---------------------+
    | BackendId | Cluster         | IP             | HeartbeatPort | BePort | HttpPort | BrpcPort | LastStartTime       | LastHeartbeat       | Alive | SystemDecommissioned | ClusterDecommissioned | TabletNum | DataUsedCapacity | AvailCapacity | TotalCapacity | UsedPct | MaxDiskUsedPct | ErrMsg | Version             |
    +-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+---------------------+
    | 10002     | default_cluster | 192.168.75.101 | 9050          | 9060   | 8040     | 8060     | 2021-06-29 14:18:36 | 2021-06-29 14:26:53 | true  | false                | false                 | 13        | 2.689 KB         | 30.565 GB     | 75.048 GB     | 59.27 % | 59.27 %        |        | 0.12.0-rc03-Unknown |
    | 10122     | default_cluster | 192.168.75.102 | 9050          | 9060   | 8040     | 8060     | 2021-06-29 14:26:48 | 2021-06-29 14:26:53 | true  | false                | false                 | 0         | .000             | 1.000 B       | .000          | 0.00 %  | 0.00 %         |        | 0.12.0-rc03-Unknown |
    | 10123     | default_cluster | 192.168.75.103 | 9050          | 9060   | 8040     | 8060     | 2021-06-29 14:26:53 | 2021-06-29 14:26:53 | true  | false                | false                 | 0         | .000             | 1.000 B       | .000          | 0.00 %  | 0.00 %         |        | 0.12.0-rc03-Unknown |
    +-----------+-----------------+----------------+---------------+--------+----------+----------+---------------------+---------------------+-------+----------------------+-----------------------+-----------+------------------+---------------+---------------+---------+----------------+--------+---------------------+
    3 rows in set (0.00 sec)
    
    mysql> show frontends;
    +----------------------------------+---------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+
    | Name                             | IP            | EditLogPort | HttpPort | QueryPort | RpcPort | Role     | IsMaster | ClusterId  | Join | Alive | ReplayedJournalId | LastHeartbeat       | IsHelper | ErrMsg |
    +----------------------------------+---------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+
    | 192.168.122.1_9010_1624848281806 | 192.168.122.1 | 9010        | 8030     | 9030      | 9020    | FOLLOWER | true     | 2117830509 | true | true  | 29923             | 2021-06-29 14:26:53 | true     |        |
    +----------------------------------+---------------+-------------+----------+-----------+---------+----------+----------+------------+------+-------+-------------------+---------------------+----------+--------+
    1 row in set (0.06 sec)
  • 相关阅读:
    12. nc/netcat 用法举例
    7. 由一道ctf学习变量覆盖漏洞
    11. 几点基于Web日志的Webshell检测思路
    约瑟夫环
    栈结构的经典算法题
    二叉查找树之二
    fork与vfork
    数组常见算法题
    赛马问题
    fibonacci 数列及其应用
  • 原文地址:https://www.cnblogs.com/ywjfx/p/14719217.html
Copyright © 2020-2023  润新知