出现的异常是:java.sql.SQLException: No suitable driver。是因为缺少mysql包:
代码中:
val alterDs: DataFrame = sparkSession.read.format("jdbc")
.option("url", url)
.option("driver","com.mysql.jdbc.Driver")->有mysql驱动
.option("dbtable", altertable)
.option("user", user)
.option("password", password)
.load()
提交命令:
spark-submit
--name OnLineLogAnalysisTest1
--class com.importpack.OnLineLogAnalysis
--master yarn
--deploy-mode cluster
--driver-memory 2G
--executor-memory 2G
--executor-cores 1
--num-executors 1
--packages org.influxdb:influxdb-java:2.5,org.apache.spark:spark-streaming-kafka-0-10_2.11:2.4.2
--jars /home/hadoop/lib/fastjson-1.2.58.jar,/home/hadoop/lib/mysql-connector-java-5.1.43-bin.jar -------->在executor中加入mysql驱动的包
--driver-class-path /home/hadoop/lib/fastjson-1.2.58.jar,/home/hadoop/lib/mysql-connector-java-5.1.43-bin.jar ------>在driver中也加入mysql驱动的包
/home/hadoop/lib/ruozetest-1.0.jar