datax的开源地址:https://github.com/alibaba/DataX
参考安装博客:https://blog.csdn.net/u014646662/article/details/82748055
https://www.cnblogs.com/zlc364624/p/12374089.html
datax下载地址:http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz
步骤:
1,环境准备:
JDK1.8
Python2.7
DataX
2,测试安装是否成功,打开命令界面
①cd (你的datax安装目录)in
②py -2 {YOUR_DATAX_HOME}/bin/datax.py {YOUR_DATAX_HOME}/job/job.json 如:py -2 E:MySoftWaredataxdataxindatax.py E:MySoftWaredataxdataxjobjob.json
如果成功:
则出现以下界面则成功,若出现乱码,输入CHCP 65001,即可解决
3,创建mysql数据表和csv文件,测试csv导入mysql表
mysql表:
CREATE TABLE `demo` ( `id` varchar(11) NOT NULL, `name` varchar(255) DEFAULT NULL, `age` varchar(255) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
csv样式:
demo.json:
{ "job": { "content": [ { "reader": { "name": "txtfilereader", "parameter": { "path": ["E:/example/demo.csv"], "encoding":"GBK", "column": [ { "index": 0, "type": "string" }, { "index": 1, "type": "string" }, { "index": 2, "type": "string" }, ], "skipHeader": "true" } }, "writer": { "name": "mysqlwriter", "parameter": { "column": [ "id", "name", "age", ], "connection": [ { "jdbcUrl": "jdbc:mysql://localhost:3306/data?useUnicode=true&characterEncoding=utf8", "table": ["demo"] } ], "password": "123456", "username": "root", "preSql":[""], "session":["set session sql_mode='ANSI'"], "writeMode":"insert" } } } ], "setting": { "speed": { "channel": "1" } } } }
使用命令:py -2 E:MySoftWaredataxdataxindatax.py E:exampledemo.json
运行结果如下:
成功将数据导入mysql中。
3,遇到的错误。
我遇到的错误有 您提供的配置信息不是合法的JSON格式: syntax error, position at 795, name fieldDelimiter
因为我json文件中加上了"fieldDelimiter":"," 这种写法不对,删除即可。