• 二千万のホテルデータ処理


    CREATE TABLE Hotel
    (
    Name varchar(255),
    CardNo varchar(255),
    Descriot varchar(255),
    CtfTp varchar(255),
    CtfId varchar(255),
    Gender varchar(255),
    Birthday varchar(255),
    Address varchar(255),
    Zip varchar(255),
    Dirty varchar(255),
    District1 varchar(255),
    District2 varchar(255),
    District3 varchar(255),
    District4 varchar(255),
    District5 varchar(255),
    District6 varchar(255),
    FirstNm varchar(255),
    LastNm varchar(255),
    Duty varchar(255),
    Mobile varchar(255),
    Tel varchar(255),
    Fax varchar(255),
    EMail varchar(255),
    Nation varchar(255),
    Taste varchar(255),
    Education varchar(255),
    Company varchar(255),
    CAddress varchar(255),
    CZip varchar(255),
    Family varchar(255),
    Version varchar(255),
    id varchar(255)
    )

    db2命令行批量导入

    IMPORT FROM 'C:XINVOICEMODEL_20121203.CSV' OF DEL

    INSERT_UPDATE INTO SNDBUSR.XINVOICEMODEL(field1, field2...);

    問題❶:

    db2 => list tables
    
    Table/View                      Schema          Type  Creation time
    ------------------------------- --------------- ----- --------------------------
    CATALOG                         DB2INST1        T     2011-10-21-16.35.38.068444
    
      1 record(s) selected.
    
    db2 => IMPORT FROM IMPORT.csv OF DEL INSERT INTO CATALOG;
    SQL3035N  The tablename parameter in the target specification is not valid.
    
    db2 =>
    解決策:

    Hi Greensburo,
    Unless you started the CLI with the -t option (changing the line terminator) the semicolon on the end of the line is the problem.  
    Drop the semicolon.  The command should work fine.(セミロンを削除すると、できるになりました。)
    Good Luck,
    Kent

    問題❷:

    C:Program FilesIBMSQLLIBBIN>db2 import from "F:hotelA-0001-200W.csv" OF DEL INSERT INTO HOTEL
    SQL3109N  The utility is beginning to load data from file
    "F:hotelA-0001-200W.csv".

    SQL3306N  An SQL error "-964" occurred while inserting a row into the table.

    SQL0964C  The transaction log for the database is full.  SQLSTATE=57011

    SQL3110N  The utility has completed processing.  "122394" rows were read from
    the input file.

    解決策:

    数据库的事务日志满了,调大以下几个数据库参数
    Log file size (4KB)                         (LOGFILSIZ)
    Number of primary log files                (LOGPRIMARY)
    Number of secondary log files               (LOGSECOND)

    image

    修改对应的参数,使用命令:

    update db cfg [fordbname] using <参数名> <参数值>

    db2 => update db cfg using logretain on

    image

    image

    image

    第一批:

    C:Program FilesIBMSQLLIBBIN>db2 import from "F:hotelA-0001-200W.csv" OF DEL INSERT INTO HOTEL
    SQL3109N  The utility is beginning to load data from file
    "F:hotelA-0001-200W.csv".

    SQL3148W  A row from the input file was not inserted into the table.  SQLCODE
    "-302" was returned.

    SQL0302N  The value of a host variable in the EXECUTE or OPEN statement is out
    of range for its corresponding use.  SQLSTATE=22001

    SQL3185W  The previous error occurred while processing data from row "799240"
    of the input file.

    SQL3114W  Some data following "CHEN" in row "936527" and column "1" was not
    loaded.

    SQL3148W  A row from the input file was not inserted into the table.  SQLCODE
    "-302" was returned.

    SQL0302N  The value of a host variable in the EXECUTE or OPEN statement is out
    of range for its corresponding use.  SQLSTATE=22001

    SQL3185W  The previous error occurred while processing data from row "1177644"
    of the input file.

    SQL3148W  A row from the input file was not inserted into the table.  SQLCODE
    "-302" was returned.

    SQL0302N  The value of a host variable in the EXECUTE or OPEN statement is out
    of range for its corresponding use.  SQLSTATE=22001

    SQL3185W  The previous error occurred while processing data from row "1541815"
    of the input file.

    SQL3148W  A row from the input file was not inserted into the table.  SQLCODE
    "-302" was returned.

    SQL0302N  The value of a host variable in the EXECUTE or OPEN statement is out
    of range for its corresponding use.  SQLSTATE=22001

    SQL3185W  The previous error occurred while processing data from row "1598633"
    of the input file.

    SQL3148W  A row from the input file was not inserted into the table.  SQLCODE
    "-302" was returned.

    SQL0302N  The value of a host variable in the EXECUTE or OPEN statement is out
    of range for its corresponding use.  SQLSTATE=22001

    SQL3185W  The previous error occurred while processing data from row "1727125"
    of the input file.

    SQL3148W  A row from the input file was not inserted into the table.  SQLCODE
    "-302" was returned.

    SQL0302N  The value of a host variable in the EXECUTE or OPEN statement is out
    of range for its corresponding use.  SQLSTATE=22001

    SQL3185W  The previous error occurred while processing data from row "1948313"
    of the input file.

    SQL3110N  The utility has completed processing.  "2000094" rows were read from
    the input file.

    SQL3221W  ...Begin COMMIT WORK. Input Record Count = "2000094".

    SQL3222W  ...COMMIT of any database changes was successful.

    SQL3149N  "2000094" rows were processed from the input file.  "2000088" rows
    were successfully inserted into the table.  "6" rows were rejected.


    Number of rows read         = 2000094
    Number of rows skipped      = 0
    Number of rows inserted     = 2000088
    Number of rows updated      = 0
    Number of rows rejected     = 6
    Number of rows committed    = 2000094

  • 相关阅读:
    Android必学——AsyncTask
    成员变量 局部变量
    Java中private、protected、public和default的区别
    实现图片的一个轮转功能
    vmware14克隆后UUID相同的解决方法
    curl
    python3 configparser模块
    python2.7系统性能监控psutil模块
    mysql5.7.22tar包安装
    通过pip3安装ipython
  • 原文地址:https://www.cnblogs.com/vonk/p/3865841.html
Copyright © 2020-2023  润新知