用SQL client 能搞通redis 维表join,我在网上还没找到,我就试验,终于在sql client里运行成功了。
我用的 “大数据羊说”写的redis connector(https://github.com/liufeiSAP/flink-study.git),flink用的flink-1.13.2_scala_2.11。
调试了几天才通的,就是各种缺jar包。
1: Redis connector的源码:
https://github.com/liufeiSAP/flink-study.git
2: Flink 版本: \flink-1.13.2_scala_2.11
3: 需要的jar包:
3: 以下的DDL,SQL都是在SQL client里执行。
创建一个源表,从csv文件读取数据
CREATE TABLE MyTable( MyField1 INT, MyField2 STRING, MyField3 INT, proctime AS PROCTIME() ) WITH ( 'connector' = 'filesystem', 'path' = '/opt/flink/lib/test.csv', 'format' = 'csv' );
csv文件内容
创建redis 维表:
CREATE TABLE dimTable ( name STRING, name1 STRING ) WITH ( 'connector' = 'redis', 'hostname' = '192.168.5.6', 'port' = '6379', 'format' = 'json', 'lookup.cache.max-rows' = '500', 'lookup.cache.ttl' = '3600', 'lookup.max-retries' = '1' );
最后执行
SELECT o.MyField1,o.MyField2, c.name, c.name1 FROM MyTable AS o LEFT JOIN dimTable FOR SYSTEM_TIME AS OF o.proctime AS c ON o.MyField2 = c.name;