pgspider griddb fdw +plv8 集成使用

以下是一个简单的griddb fdw 与plv8 的集成，同时添加了node require 模块的支持，可以实现灵活的系统扩展

pgspider plv8 docker镜像制作

• dockerfile

FROM dalongrong/pgspider:base as build

WORKDIR /app

ENV PLV8_VERSION=v2.3.12

ENV PLV8_VERSION_1=2.3.12

RUN apt-get update && apt-get install -y ca-certificates python curl libc++-dev libc++abi-dev pkg-config

RUN curl -o ${PLV8_VERSION}.tar.gz -SL "https://github.com/plv8/plv8/archive/$PLV8_VERSION.tar.gz" 

    && tar -xzf ${PLV8_VERSION}.tar.gz && mv plv8-${PLV8_VERSION_1} plv8 && cp -rf plv8 /app/postgresql-11.6/contrib/plv8_fdw 

    && cd /app/postgresql-11.6/contrib/plv8_fdw && make static && make install

​

FROM debian:stretch-slim

ENV GOSU_VERSION 1.11

RUN apt-get update && apt-get install -y wget libc++1 libreadline-dev

# explicitly set user/group IDs

RUN set -eux; 

 groupadd -r postgres --gid=999; 

# https://salsa.debian.org/postgresql/postgresql-common/blob/997d842ee744687d99a2b2d95c1083a2615c79e8/debian/postgresql-common.postinst#L32-35

 useradd -r -g postgres --uid=999 --home-dir=/var/lib/postgresql --shell=/bin/bash postgres; 

# also create the postgres user's home directory with appropriate permissions

# see https://github.com/docker-library/postgres/issues/274

 mkdir -p /var/lib/postgresql; 

 chown -R postgres:postgres /var/lib/postgresql

​

RUN wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" 

   && chmod +x /usr/local/bin/gosu 

   && gosu nobody true

# make the "en_US.UTF-8" locale so postgres will be utf-8 enabled by default

RUN set -eux; 

   if [ -f /etc/dpkg/dpkg.cfg.d/docker ]; then 

   # if this file exists, we're likely in "debian:xxx-slim", and locales are thus being excluded so we need to remove that exclusion (since we need locales)

   grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; 

   sed -ri '//usr/share/locale/d' /etc/dpkg/dpkg.cfg.d/docker; 

   ! grep -q '/usr/share/locale' /etc/dpkg/dpkg.cfg.d/docker; 

   fi; 

   apt-get update; apt-get install -y locales; rm -rf /var/lib/apt/lists/*; 

   localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8

ENV LANG en_US.utf8

​

# install "nss_wrapper" in case we need to fake "/etc/passwd" and "/etc/group" (especially for OpenShift)

# https://github.com/docker-library/postgres/issues/359

# https://cwrap.org/nss_wrapper.html

RUN set -eux; 

   apt-get update; 

   apt-get install -y --no-install-recommends libnss-wrapper; 

   rm -rf /var/lib/apt/lists/*

​

RUN mkdir /docker-entrypoint-initdb.d

COPY --from=build /usr/local/pgspider /usr/local/pgspider

RUN sed -ri "s!^#?(listen_addresses)s*=s*S+.*!1 = '*'!" /usr/local/pgspider/share/postgresql/postgresql.conf.sample; 

   grep -F "listen_addresses = '*'" /usr/local/pgspider/share/postgresql/postgresql.conf.sample

RUN mkdir -p /var/run/postgresql && chown -R postgres:postgres /var/run/postgresql && chmod 2777 /var/run/postgresql

ENV PATH $PATH:/usr/local/pgspider/bin

ENV PGDATA /var/lib/postgresql/data

RUN mkdir -p "$PGDATA" && chown -R postgres:postgres "$PGDATA" && chmod 777 "$PGDATA"

VOLUME /var/lib/postgresql/data

COPY docker-entrypoint.sh /usr/local/bin/

RUN ln -s usr/local/bin/docker-entrypoint.sh / # backwards compat

ENTRYPOINT ["docker-entrypoint.sh"]

​

EXPOSE 5432

CMD ["postgres"]

说明，已经push docker hub 了，同时添加了mysql,sqlite,oracle,tds,influxdb,mongodb 等的支持

使用

• docker-compose 环境

version: "3"

services: 

  pgspider-griddb:

    image: dalongrong/pgspider:griddb-4.2-plv8

    environment: 

    - "POSTGRES_PASSWORD=dalong"

    ports: 

    - "5432:5432"

  griddb-java:

    build: 

      context: ./

      dockerfile: ./Dockerfile-java

    environment: 

    - "MEMBER_ADDR=griddb:10001"

  griddb:

    build: ./

    volumes: 

    - "./log:/var/lib/gridstore/log"

    ports:

    - "10001:10001"

    - "10010:10010" 

    - "10020:10020" 

    - "10030:10030" 

    - "10040:10040"

    - "10050:10050"

    - "10080:10080"

    - "20001:20001"

• 项目说明
  包含了griddb ,一个java 应用，写入demo 数据的，一个包含plv8 以及griddb fdw 的pg，以及nodejs应用提供node require
  支持以及数据查询，详细代码参考https://github.com/rongfengliang/plv8-griddb-learning
• 使用
  启动griddb 以及pg

 

docker-compose up -d griddb pgspider-griddb

• 启动java 应用，初始化数据

docker-compose up griddb-java

• 使用扩展griddb

griddb fdw:

CREATE EXTENSION griddb_fdw;

CREATE SERVER griddb_svr FOREIGN DATA WRAPPER griddb_fdw OPTIONS(notification_member 'griddb:10001',clustername 'defaultCluster');

CREATE USER MAPPING FOR public SERVER griddb_svr OPTIONS(username 'admin', password 'admin');

IMPORT FOREIGN SCHEMA griddb_schema FROM SERVER griddb_svr INTO public;

select * from col01;

• 集成plv8

cd node-app && yarn && node app.js 

• 修改session 配置重启pg
  为了方便每个用户集成pg 的plv8 扩展的require 模块，需要重启

 

ALTER DATABASE postgres SET "plv8.start_proc" TO "v8.plv8_init";

• 创建函数

CREATE or replace FUNCTION shortid() RETURNS text AS

$$

   const shortid = require('shortid');

   const result = shortid.generate();

   return result;

$$

LANGUAGE plv8;

• 插入数据
  nodejs 应用

 

cd node-app && yarn && node index.js 

psql 管理工具

说明你

关于griddb 以及plv8 的可以参考官方文档，或者我写的一些简单的文章

参考资料

https://github.com/rongfengliang/plv8-griddb-learning
https://www.cnblogs.com/rongfengliang/category/1584238.html
https://www.cnblogs.com/rongfengliang/category/1642606.html
https://www.cnblogs.com/rongfengliang/category/1641930.html
https://hub.docker.com/repository/docker/dalongrong/pgspider