Scala操作外部数据

Posted green-frog-2019

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Scala操作外部数据相关的知识,希望对你有一定的参考价值。

Scala操作外部数据:

1、操作文件
2、操作XML
3、操作mysql


读取文件:

object FileApp
  def main(args: Array[String]): Unit =

    //system file
    val file = Source.fromFile("Users/rocky/imooc/hello.txt") (scala.io.Codec.UTF8)

    def readLine(): Unit =
      for(line <- file.getLines()) //一行一行的读取
        println(line)
      
    

    readLine()

    //URL
    def readNet(): Unit =
      val file = Source.fromURL("http://www.baidu.com")
      for(line <- file.getLines()) //一行一行的读取
        println(line)
      
    
  

在配置文件里引入mysql的依赖:

<dependcency>
  <groupId>mysql</groupId>
  <artifactId>mysql-connector-java</artifactId>
  <version>5.1.45</version>
</dependcency>

操作MySQL数据库:
object MySQLApp extends App

  val url = "jdbc:mysql://localhost:3306/mysql"
  val username = "root"
  val password = "root"

  var connection:Connection = null
  try

    // make the connection
    classOf[com.mysql.jdbc.Driver]

    //拿到连接
    val connection = DriverManager.getConnection(url, username, password)
    //create the statement, and run the select query
    val statement = connection.createStatement()
    val resultSet = statement.executeQuery("select host,user from user")
    while(resultSet.next())
      val host = resultSet.getString("host")
      val user = resultSet.getString("user")

      println(s"$host, $user")
     catch
      case e:Exception => e.printStackTrace()
     finally
      //free
      if(connection == null)
        connection.close()
      
    
  

操作XML文件:

object XMLApp extends App

  //loadXML()
  readXMLAttr()

  //第一种方式:load(ClassPath)
  def loadXML(): Unit =
    //val xml = XML.load(this.getClass.getClassLoader.getResource("test.xml"))
    //println(xml)


    //第二种方式:load(is: InputStream)
    //val xml = XML.load(new FileInputStream("/Users/rocky/source/scala-train/src/main/resources/test.xml"))

    //第三种方式:load(reader)
    //val xml = XML.load(new InputStreamReader(new FileInputStream("/Users/rocky/source/scala-train/src/main/resources/test.xml"))
  

  //读取XML文件里的字段的值
  def readXMLAttr(): Unit =
    val xml = XML.load(this.getClass.getClassLoader.getResource("PK.xml"))

    //header/field
    val headerField = xml \ "header" \ "field"
    println(headerField)

    //all field
    val fields = xml \\ "field"
    for (field <- fields)
      println(field)
    

    //header/field/name
    //val filedAttributes = (xml \ "header" \ "field").map(_ \ "@name")
    val filedAttributes = (xml \ "header" \ "field" \\ "@name")
    for (filedAttribute <- filedAttributes)
      println(filedAttribute)
    

    //name="Logon" message
    //val filters = (xml \\ "message").filter(_.attribute("name").exists(_.text.equals("Logon")))
    val filters = (xml \\ "message").filter(x => ((x \ "@name").text).equals("Logon"))
    for (filter <- filters)
      println(filter)
    


    // header/field/name content
    (xml \ "header" \ "field").map(x => (x \ "@name", x.text, x \ "@required")) .foreach(println)
  

以上是关于Scala操作外部数据的主要内容,如果未能解决你的问题,请参考以下文章

Scala编程进阶

scala为什么要清理闭包

类型安全配置:从打包的 scala 应用程序的外部路径加载附加配置

Spark Scala创建外部配置单元表不使用位置作为变量

如何从代码外部提供spark / scala中的模式

Spark scala.collection.immutable.$colon$colon 不是字符串模式的有效外部类型