大数据处理之Dinky篇

官网地址 Dinky 是一个开箱即用的一站式实时计算平台，以 Apache Flink 为基础，连接数据湖仓等众多框架，致力于流批一体和湖仓一体的建设与实践。一、安装部署1、linux单机安装Dinky1.1、下载 点击gitee下载地址比官网下载快10倍1.2、引入jar包阿里云仓库服务A、下载/拷贝对应版本flink的lib/*.jar到/juxin/dinky/lib/下 (log4j之类的包不要拷贝，不然会冲突)1. flink-table-runtime-1.19.3.jar2. flink-table-planner-loader-1.19.3.jar3. flink-table-api-java-uber-1.19.3.jar4. flink-scala_2.12-1.19.3.jar5. flink-json-1.19.3.jar6. flink-dist-1.19.3.jar7. flink-csv-1.19.3.jar8. flink-connector-files-1.19.3.jar9. flink-cep-1.19.3.jar提示：复制上述jar至dinky/lib目录时，**不要复制log4j相关jar包**，避免日志冲突。B、flink-table-runtime-1.19.3.jar、flink-table-planner-blink_2.11-1.93.jar没有这两个包会报错Caused by: java.lang.NoClassDefFoundError: org/apache/flink/table/planner/parse/ExtendedParserC、拷贝mysql-connector-j-8.0.33.jar包到/juxin/dinky-release-1.19-1.2.4/lib下1.3、配置文件修改 Dinky 配置文件，选择默认数据源为 mysqlcd /opt/dinky/config/vim application.yml# 修改 Dinky 所使用的数据库类型为 mysqlspring: application: name: Dinky profiles: # The h2 database is used by default. If you need to use other databases, please set the configuration active to: mysql, currently supports [mysql, pgsql, h2] # If you use mysql database, please configure mysql database connection information in application-mysql.yml # If you use pgsql database, please configure pgsql database connection information in application-pgsql.yml # If you use the h2 database, please configure the h2 database connection information in application-h2.yml, # note: the h2 database is only for experience use, and the related data that has been created cannot be migrated, please use it with cautionnbs