pgspider zombodb+opendistro 集成

Posted 2021-03-10 rongfengliang-荣锋亮

zombodb 默认对于es的连接是没有密码的，很不安全，可选的方式，基于nginx+basci auth
使用es 的X-Pack，使用amazon的opendistro 也是一个很不错的方案,以下是一个简单测试，同时
会有关于使用basic auth 模式的连接配置

注意，测试使用的pg zombodb docker 镜像使用zombodb 4.0 构建，会有es 兼容的问题，所以最好选择好opendistro 版本

环境准备

• docker-compose 文件

  注意es 禁用了tls 的认证，使用了单机模式

 

version: "3"

services: 

  pg-zombodb:

    image: dalongrong/pgspider:zombodb-plv8

    ports:

    - "5432:5432"

    environment: 

    - "POSTGRES_PASSWORD=dalong"

  elasticsearch:

    image: amazon/opendistro-for-elasticsearch:0.10.0

    ports: 

    - "9200:9200"

    environment:

      - "discovery.type=single-node"

      - "http.host=0.0.0.0"

      - "opendistro_security.ssl.http.enabled=false"

      - "cluster.name=odfe-cluster"

      - "transport.host=0.0.0.0"

      - "network.host=0.0.0.0"

      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"

使用

• 启动服务

docker-compose up -d

• 启用扩展

CREATE EXTENSION zombodb;

// 创建表

CREATE TABLE products (

  id SERIAL8 NOT NULL PRIMARY KEY,

  name text NOT NULL,

  keywords varchar(64)[],

  short_summary text,

  long_description zdb.fulltext, 

  price bigint,

  inventory_count integer,

  discontinued boolean default false,

  availability_date date

);

// 添加索引，同时使用basic auth 模式连接es

CREATE INDEX idxproducts 

                   ON products 

                USING zombodb ((products.*))

                 WITH (url=\'http://admin:admin@elasticsearch:9200/\');

 

 

• 添加测试数据

INSERT INTO "public"."products"("id","name","keywords","short_summary","long_description","price","inventory_count","discontinued","availability_date")

VALUES

(1,E\'Magical Widget\',E\'{magical,widget,round}\',E\'A widget that is quite magical\',E\'Magical Widgets come from the land of Magicville and are capable of things you can\\\'t imagine\',9900,42,FALSE,E\'2015-08-31\'),

(2,E\'Baseball\',E\'{baseball,sports,round}\',E\'It\\\'s a baseball\',E\'Throw it at a person with a big wooden stick and hope they don\\\'t hit it\',1249,2,FALSE,E\'2015-08-21\'),

(3,E\'Telephone\',E\'{communication,primitive,"alexander graham bell"}\',E\'A device to enable long-distance communications\',E\'Use this to call your friends and family and be annoyed by telemarketers. Long-distance charges may apply\',1899,200,FALSE,E\'2015-08-11\'),

(4,E\'Box\',E\'{wooden,box,"negative space",square}\',E\'rong\',E\'A wooden container that will eventually rot away. Put stuff it in (but not a cat).\',17000,0,TRUE,E\'2015-07-01\');

• 查询

select * from products WHERE products ==> \'rong\';

效果

 

 

说明

amazon的opendistro 是一个很不错的es 替代，功能强大，安全性以及周边也都很不错，加上zombodb真的很不错

参考资料

https://opendistro.github.io/for-elasticsearch-docs/docs/security-configuration/tls/ 
https://opendistro.github.io/for-elasticsearch-docs/version-history/ 
https://github.com/zombodb/zombodb/releases 
https://hub.docker.com/repository/docker/dalongrong/pgspider 
https://www.cnblogs.com/rongfengliang/category/1433947.html 
https://www.cnblogs.com/rongfengliang/category/1641930.html 
https://hub.docker.com/repository/docker/dalongrong/pgspider 
https://github.com/rongfengliang/zombodb_opendistro-for-elasticsearch