• spark操作hive方式(scala)


    第一种方式:

    def operatorHive: Unit = {
        Class.forName("org.apache.hive.jdbc.HiveDriver")
        val url = "jdbc:hive2://192.168.2.xxx:10000"
        val connection: Connection = DriverManager.getConnection(url, "root", "diagbotkwz@123")
        val createStatement: Statement = connection.createStatement()
        val query: ResultSet = createStatement.executeQuery("select * from diagbot.ord_lis_trend limit 2")
        while (query.next()) {
          println(query.getString(1))
        }
      }

    第二种方式:

    object SparkOperaterHive {
      val sparkConf: SparkConf = new SparkConf().setAppName(SparkOperaterHive.getClass.getSimpleName)
      val sparkSession: SparkSession = SparkSession.builder().config(sparkConf).enableHiveSupport().getOrCreate()
      val sc: SparkContext = sparkSession.sparkContext
      val sqlContext: SQLContext = sparkSession.sqlContext
      
      def main(args: Array[String]) {
       
        import sparkSession.implicits._
        val sql1: DataFrame = sparkSession.sql("select * from janggan.diagnosismedication")
        val properties: Properties = new Properties()
        properties.put("user", "root")
        properties.put("password", "diagbot@20180822")
        properties.put("driver", "com.mysql.jdbc.Driver")
        //    sql1.write.mode(SaveMode.Append).jdbc(url,"doc_info_hive",properties)
        println("总数为:" + sql1.count())
        println("sddhdj" + sql1.columns(1))
    
        sparkSession.stop()
      }
    }
  • 相关阅读:
    2019-10-28-开源项目
    2018-8-10-win10-uwp-MetroLog-入门
    2018-5-20-C#-BBcode-转-Markdown
    2018-8-10-win10-UWP-序列化
    2018-2-13-win10-uwp-BadgeLogo-颜色
    2019-1-25-WPF-ListBox-的选择
    2019-1-5-Windows-的-Pen-协议
    android studio打印
    Java 基本数据类型
    FreeRTOS 任务通知模拟计数型信号量
  • 原文地址:https://www.cnblogs.com/kwzblog/p/10180174.html
Copyright © 2020-2023  润新知