• pgloader 学习(七) 从归档文件加载数据


    我们可以直接从zip,tar,gzip 文件获取内容

    command file 参考格式

    LOAD ARCHIVE
       FROM /Users/dim/Downloads/GeoLiteCity-latest.zip
       INTO postgresql:///ip4r
       BEFORE LOAD
         DO $$ create extension if not exists ip4r; $$,
            $$ create schema if not exists geolite; $$,
         EXECUTE 'geolite.sql'
       LOAD CSV
            FROM FILENAME MATCHING ~/GeoLiteCity-Location.csv/
                 WITH ENCODING iso-8859-1
                 (
                    locId,
                    country,
                    region null if blanks,
                    city null if blanks,
                    postalCode null if blanks,
                    latitude,
                    longitude,
                    metroCode null if blanks,
                    areaCode null if blanks
                 )
            INTO postgresql:///ip4r?geolite.location
                 (
                    locid,country,region,city,postalCode,
                    location point using (format nil "(~a,~a)" longitude latitude),
                    metroCode,areaCode
                 )
            WITH skip header = 2,
                 fields optionally enclosed by '"',
                 fields escaped by double-quote,
                 fields terminated by ','
      AND LOAD CSV
            FROM FILENAME MATCHING ~/GeoLiteCity-Blocks.csv/
                 WITH ENCODING iso-8859-1
                 (
                    startIpNum, endIpNum, locId
                 )
            INTO postgresql:///ip4r?geolite.blocks
                 (
                    iprange ip4r using (ip-range startIpNum endIpNum),
                    locId
                 )
            WITH skip header = 2,
                 fields optionally enclosed by '"',
                 fields escaped by double-quote,
                 fields terminated by ','
       FINALLY DO
         $$ create index blocks_ip4r_idx on geolite.blocks using gist(iprange); $$;

    source 指南: from

    文件名或者一个http 的url,如果是一个url,在处理之前会先下载到本地

    sub command

    • command [AND command ...]
      目前仅支持CSV,“FIXED”和DBF命令

    Final SQL Commands

    • FINALLY DO
      数据加载完成之后执行的sql 命令,比如create index

    参考资料

    https://pgloader.readthedocs.io/en/latest/ref/archive.html

  • 相关阅读:
    case when的使用
    docker 修改容器时间 和时区 的方法 (linux也能用)
    canal笔记
    jenkins集成git,输入分支构建的方法(还没试,用到再看看)
    unittest官网和别人的翻译(有空看看)
    靶机
    CSharp: Reflection in donet 6
    Python: Memento Pattern
    Python: Visitor Pattern
    Python: Command Pattern
  • 原文地址:https://www.cnblogs.com/rongfengliang/p/11019307.html
Copyright © 2020-2023  润新知