• troubleshooting-When importing query results in parallel, you must specify --split-by.


    原因分析

    -m 4 导数命令中map task number=4,当-m 设置的值大于1时,split-by必须设置字段(需要是 int 类型的字段),如果不是 int类型的字段,则需要加上参数
    -Dorg.apache.sqoop.splitter.allow_text_splitter=true
    例子:
    sqoop import -Dorg.apache.sqoop.splitter.allow_text_splitter=true
    --connect ${conn_str}
    --username ${db_username}
    --password ${db_password}

    解决办法

    并行导入数据需要加上 --split-by columnName 属性。

    异常日志

    18/09/17 14:23:58 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7-cdh6.0.0
    18/09/17 14:23:58 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
    When importing query results in parallel, you must specify --split-by.
    Try --help for usage instructions.
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Table not found user_base_delta

  • 相关阅读:
    node 中的 异步地狱回调
    node 同步和异步的概念
    【Node】File System
    阅读《软技能:代码之外的生存指南》读书笔记
    整理前端学习资料以便日后查看
    【css】单选框和复选框文字垂直居中问题
    [CSS]图片与文字对齐问题--摘自张鑫旭博客
    百度首页换一换功能js实现
    个人加分项
    开课第十五周周总结
  • 原文地址:https://www.cnblogs.com/chwilliam85/p/9693268.html
Copyright © 2020-2023  润新知