Scala Macros － scalamela 1.x，inline-meta annotations

Posted 2020-08-22 雪川大虫

  在上期讨论中我们介绍了Scala Macros，它可以说是工具库编程人员不可或缺的编程手段，可以实现编译器在编译源代码时对源代码进行的修改、扩展和替换，如此可以对用户屏蔽工具库复杂的内部细节，使他们可以用简单的声明方式，通过编译器自动产生铺垫代码来实现工具库中各种复杂的类型、对象及方法函数的构建。虽然Def Macros可能具备超强的编程功能，但同时使用者也普遍认为它一直存有着一些严重的诟病：包括用法复杂、容易犯错、运算行为难以预测以及没用完善的集成开发环境工具（IDE）支持等。这些恶评主要是因为Def Macros和编译器scalac捆绑的太紧，使用者必须对编译器的内部运作原理和操作函数有比较深刻的了解。加之Def Macros向用户提供的api比较复杂且调用繁琐，其中比较致命的问题就是与scalac的紧密捆绑了：因为Def Macros还只是一项实验性功能，没有scala语言规范文件背书，肯定会面临升级换代。而且scala本身也面临着向2.12版本升级的情况，其中dotty就肯定是scalac的替代编译器。Scalameta是根据scala语言规范SIP-28-29-Inline-Macros由零重新设计的Macros编程工具库。主要目的就是为了解决Def Macros所存在的问题，而且Jetbrains的IntelliJ IDEA 2016.3 EAP对Scalameta已经有了比较好的支持，能为使用者带来更简单、安全的Macros编程工具。

  我在介绍了Slick之后立即转入Scala Macros是有一些特别目的的。研究FRM Slick乃至学习泛函编程的初衷就是希望能为传统的OOP编程人员提供更简单易用的泛函库应用帮助，使他们无须对函数式编程模式有太深刻了解也能使用由函数式编程模式所开发的函数库。实现这个目标的主要方式就是Macros了。希望通过Macros的产生代码功能把函数库的泛函特性和模式屏蔽起来，让用户能用他们习惯的方式来定义函数库中的类型对象、调用库中的方法函数。

  Macros功能实现方式（即编译时的源代码扩展compile time expansion）由两个主要部分组成：一是在调用时扩展（on call expansion），二是申明时扩展即注释（annotation）。这两种方式我们在上一篇讨论里都一一做了示范。通过测试发现，Scalameta v1.x只支持注释方式。这事动摇了我继续探讨的意愿：试想如果没了”Implicit Macros“，“Extractor Macros“这些模式，会损失多少理想有趣的编码方式。通过与Scalameta作者沟通后得知他们将会在Scalameta v2.x中开始支持全部两种模式，因此决定先介绍一下Scalameta v1.x，主要目的是让大家先熟悉了解Scalameta新的api和使用模式。我们可以把上次Def Macros的Macros Annotations示范例子在Scalameta里重新示范一遍来达到这样的目的。

  虽然Scalameta是从头设计的，但是它还是保留了许多Def Macros的思想，特别是沿用了大部分scala-reflect的quasiquote模式。与Def Macros运算原理相同，Scalameta的Macros扩展也是基于AST（abstract syntax tree）由编译器运算产生的，因此Macros申明必须先完成编译，所以我们还是沿用了上一篇讨论中的build.sbt，保留项目结构，及demos对macros的这种依赖关系。

 1 name := "learn-scalameta"
 2 
 3 val commonSettings = Seq(
 4   version := "1.0" ,
 5   scalaVersion := "2.11.8",
 6   scalacOptions ++= Seq("-deprecation", "-feature"),
 7   resolvers += Resolver.sonatypeRepo("snapshots"),
 8   addCompilerPlugin(
 9     "org.scalameta" % "paradise" % "3.0.0-M5" cross CrossVersion.full),
10   scalacOptions += "-Xplugin-require:macroparadise"
11 
12 )
13 val macrosSettings = Seq(
14   libraryDependencies += "org.scalameta" %% "scalameta" % "1.3.0",
15   libraryDependencies +=  "org.scalatest" %% "scalatest" % "3.0.1" % "test"
16 )
17 lazy val root = (project in file(".")).aggregate(macros, demos)
18 
19 lazy val macros  = project.in(file("macros")).
20   settings(commonSettings : _*).
21   settings(macrosSettings : _*)
22 
23 lazy val demos  = project.in(file("demos")).settings(commonSettings : _*).dependsOn(macros)

下面我们先用一个最简单的例子来开始了解Scalameta Macros Annotations：

1 object MacroAnnotDemo extends App {
2 
3   @Greetings object Greet {
4     def add(x: Int, y: Int) = println(x + y)
5   }
6 
7   Greet.sayHello("John")
8   Greet.add(1,2)
9 }

这里的注释@Greetings代表被注释对象Greet将会被扩展增加一个sayHello的函数。我们看看这个注释的实现方式：

 1 import scala.meta._
 2 
 3 class Greetings extends scala.annotation.StaticAnnotation {
 4     inline def apply(defn: Any): Any = meta {
 5       defn match {
 6         case q"object $name {..$stats}" => {
 7           q"""
 8               object $name {
 9                 def sayHello(msg: String): Unit = println("Hello," + msg)
10                 ..$stats
11               }
12             """
13         }
14         case _ => abort("annottee must be object!")
15       }
16     }
17 }

首先，我们看到这段源代码表达方式直接了许多：只需要import scala.meta，没有了blackbox、whitebox、universe这些imports。特别是避免了对blackbox.Context和whitebox.Context这些复杂运算域的人为判定。quasiquote的使用没有什么变化。直观上Macros编程简单了，实际上编写的Macros程序能更安全稳定的运行。

我们再重复演示方法注释（method annotation）的实现方法：

 1 class Benchmark extends scala.annotation.StaticAnnotation {
 2   inline def apply(defn: Any): Any = meta {
 3     defn match {
 4       case q"..$mod def $name[..$tparams](...$args): $rtpe = $body" =>
 5         q"""
 6             ..$mod def $name[..$tparams](...$args): $rtpe = {
 7             val start = System.nanoTime()
 8             val result = $body
 9             val end = System.nanoTime()
10             println(${name.toString} + " elapsed time = " + (end - start) + "ns")
11             result
12            }
13           """
14       case _ => abort("Fail to expand annotation Benchmark!")
15     }
16   }
17 }

还是固定格式。只是quasiquote的调用组合变化。用下面方法调用测试：

1   @Benchmark
2   def calcPow(x: Double, y: Double) = {
3     val z = x + y
4     math.pow(z,z)
5   }
6 
7   println(calcPow(4.2, 8.9))

在下面这个例子里我们在注释对象中增加main方法（未extends App的对象）：

 1 import scala.meta.Ctor.Call
 2 class main extends scala.annotation.StaticAnnotation {
 3   inline def apply(defn: Any): Any = meta {
 4     def abortIfObjectAlreadyExtendsApp(ctorcalls: scala.collection.immutable.Seq[Call], objectName: Term) = {
 5       val extendsAppAlready = ctorcalls.map(_.structure).contains(ctor"App()".structure)
 6       if (extendsAppAlready){
 7         abort(s"$objectName already extends App")
 8       }
 9     }
10     defn match {
11       case q"..$mods object $name extends $template" => template match {
12         case template"{ ..$stats1 } with ..$ctorcalls { $param => ..$stats2 }" =>
13           abortIfObjectAlreadyExtendsApp(ctorcalls, name)
14           val mainMethod = q"def main(args: Array[String]): Unit = { ..$stats2 }"
15           val newTemplate = template"{ ..$stats1 } with ..$ctorcalls { $param => $mainMethod }"
16 
17           q"..$mods object $name extends $newTemplate"
18       }
19       case _ => abort("@main can be annotation of object only")
20     }
21   }
22 }

下面这个是case class的注释示例：效果是添加一个从case class转Map的类型转换函数toMap：

 1 @compileTimeOnly("@Mappable not expanded")
 2 class Mappable extends StaticAnnotation {
 3   inline def apply(defn: Any): Any = meta {
 4     defn match {
 5       case q"..$mods class $tname[..$tparams] (...$paramss) extends $template" =>
 6         template match {
 7           case template"{ ..$stats } with ..$ctorcalls { $param => ..$body }" => {
 8             val expr = paramss.flatten.map(p => q"${p.name.toString}").zip(paramss.flatten.map{
 9               case param"..$mods $paramname: $atpeopt = $expropt" => paramname
10             }).map{case (q"$paramName", paramTree) => {
11               q"${Term.Name(paramName.toString)} -> ${Term.Name(paramTree.toString)}"
12             }}
13 
14             val resultMap = q"Map(..$expr)"
15 
16             val newBody = body :+ q"""def toMap: Map[String, Any] = $resultMap"""
17             val newTemplate = template"{ ..$stats } with ..$ctorcalls { $param => ..$newBody }"
18 
19             q"..$mods class $tname[..$tparams] (...$paramss) extends $newTemplate"
20           }
21         }
22       case _ => throw new Exception("@Mappable can be annotation of class only")
23     }
24   }
25 }

可以用下面的数据进行测试：

1   @Mappable
2   case class Car(color: String, model: String, year: Int, owner: String){
3     def turnOnRadio = {
4       "playing"
5     }
6   }
7 
8   val newCarMap = Car("Silver", "Ford", 1998, "John Doe").toMap
9   println(newCarMap)

在下面这个例子里示范了如何使用注释参数：

 1 import scala.util.Try
 2 @compileTimeOnly("@RetryOnFailure not expanded")
 3 class RetryOnFailure(repeat: Int) extends scala.annotation.StaticAnnotation {
 4   inline def apply(defn: Any): Any = meta {
 5     defn match {
 6       case q"..$mods def $name[..$tparams](...$paramss): $tpeopt = $expr" => {
 7         val q"new $_(${arg})" = this
 8         val repeats = Try(arg.toString.toInt).getOrElse(abort(s"Retry on failure takes number as parameter"))
 9 
10         val newCode =
11           q"""..$mods def $name[..$tparams](...$paramss): $tpeopt = {
12                 import scala.util.Try
13 
14                 for( a <- 1 to $repeats){
15                   val res = Try($expr)
16                   if(res.isSuccess){
17                     return res.get
18                   }
19                 }
20 
21                 throw new Exception("Method fails after "+$repeats + " repeats")
22               }
23             """
24         newCode
25       }
26       case _ => abort("@RetryOnFailure can be annotation of method only")
27     }
28   }
29 }

具体使用方法如下：

 object utils {
    def methodThrowingException(random: Int): Unit = {
      if(random%2 == 0){
        throw new Exception(s"throwing exception for ${random}")
      }
    }
  }
  import scala.util.Random
  @RetryOnFailure(20) def failMethod[String](): Unit = {
    val random = Random.nextInt(10)
    println("Retrying...")
    utils.methodThrowingException(random)
  }

顺便也把上次的那个TalkingAnimal重新再写一下：

 1 class TalkingAnimal(voice: String) extends StaticAnnotation {
 2   inline def apply(defn: Any): Any = meta {
 3     defn match {
 4       case q"..$mods class $tname[..$tparams] (...$paramss) extends $template" =>
 5         template match {
 6           case template"{ ..$stats } with ..$ctorcalls { $param => ..$body }" => {
 7             val q"new $_(${arg})" = this
 8             val sound = arg.toString()
 9             val animalType = tname.toString()
10             val newBody = body :+
11               q""" def sayHello: Unit =
12                      println("Hello, I‘m a " + $animalType +
13                     " and my name is " + name + " " + $sound+ "...")
14               """
15             val newTemplate =template"{ ..$stats } with ..$ctorcalls { $param => ..$newBody }"
16             q"..$mods class $tname[..$tparams] (...$paramss) extends $newTemplate"
17           }
18         }
19       case _ => abort("Error: expanding TalkingAnimal!")
20     }
21   }
22 }

对比旧款Def Macros可以发现quasiquote的语法还是有变化的，比如拆分class定义就需要先拆出template。Scalameta重新定义了新的quasiquote,另外注释对象参数的运算方法也有所不同，这是因为Scalameta的AST新设计的表达结构。

测试运算如下：

 1   trait Animal {
 2     val name: String
 3   }
 4   @TalkingAnimal("wangwang")
 5   case class Dog(val name: String) extends Animal
 6 
 7   @TalkingAnimal("miaomiao")
 8   case class Cat(val name: String) extends Animal
 9 
10   //@TalkingAnimal("")
11   //case class Carrot(val name: String)
12   //Error:(12,2) Annotation TalkingAnimal only apply to Animal inherited! @TalingAnimal
13   Dog("Goldy").sayHello
14   Cat("Kitty").sayHello

下面是本次讨论中的完整示范源代码：

注释实现源代码：

  1 import scala.meta._
  2 class Greetings extends scala.annotation.StaticAnnotation {
  3     inline def apply(defn: Any): Any = meta {
  4       defn match {
  5         case q"object $name {..$stats}" => {
  6           q"""
  7               object $name {
  8                 def sayHello(msg: String): Unit = println("Hello," + msg)
  9                 ..$stats
 10               }
 11             """
 12         }
 13         case q"object $name extends $parent {..$stats}" => {
 14             q"""
 15               object $name extends $parent {
 16                 def sayHello(msg: String): Unit = println("Hello," + msg)
 17                 ..$stats
 18               }
 19             """
 20         }
 21         case _ => abort("annottee must be object!")
 22       }
 23     }
 24 }
 25 
 26 class Benchmark extends scala.annotation.StaticAnnotation {
 27   inline def apply(defn: Any): Any = meta {
 28     defn match {
 29       case q"..$mod def $name[..$tparams](...$args): $rtpe = $body" =>
 30         q"""
 31             ..$mod def $name[..$tparams](...$args): $rtpe = {
 32             val start = System.nanoTime()
 33             val result = $body
 34             val end = System.nanoTime()
 35             println(${name.toString} + " elapsed time = " + (end - start) + "ns")
 36             result
 37            }
 38           """
 39       case _ => abort("Fail to expand annotation Benchmark!")
 40     }
 41   }
 42 }
 43 
 44 import scala.meta.Ctor.Call
 45 class main extends scala.annotation.StaticAnnotation {
 46   inline def apply(defn: Any): Any = meta {
 47     def abortIfObjectAlreadyExtendsApp(ctorcalls: scala.collection.immutable.Seq[Call], objectName: Term) = {
 48       val extendsAppAlready = ctorcalls.map(_.structure).contains(ctor"App()".structure)
 49       if (extendsAppAlready){
 50         abort(s"$objectName already extends App")
 51       }
 52     }
 53     defn match {
 54       case q"..$mods object $name extends $template" => template match {
 55         case template"{ ..$stats1 } with ..$ctorcalls { $param => ..$stats2 }" =>
 56           abortIfObjectAlreadyExtendsApp(ctorcalls, name)
 57           val mainMethod = q"def main(args: Array[String]): Unit = { ..$stats2 }"
 58           val newTemplate = template"{ ..$stats1 } with ..$ctorcalls { $param => $mainMethod }"
 59 
 60           q"..$mods object $name extends $newTemplate"
 61       }
 62       case _ => abort("@main can be annotation of object only")
 63     }
 64   }
 65 }
 66 import scala.annotation.{StaticAnnotation, compileTimeOnly}
 67 @compileTimeOnly("@Mappable not expanded")
 68 class Mappable extends StaticAnnotation {
 69   inline def apply(defn: Any): Any = meta {
 70     defn match {
 71       case q"..$mods class $tname[..$tparams] (...$paramss) extends $template" =>
 72         template match {
 73           case template"{ ..$stats } with ..$ctorcalls { $param => ..$body }" => {
 74             val expr = paramss.flatten.map(p => q"${p.name.toString}").zip(paramss.flatten.map{
 75               case param"..$mods $paramname: $atpeopt = $expropt" => paramname
 76             }).map{case (q"$paramName", paramTree) => {
 77               q"${Term.Name(paramName.toString)} -> ${Term.Name(paramTree.toString)}"
 78             }}
 79 
 80             val resultMap = q"Map(..$expr)"
 81 
 82             val newBody = body :+ q"""def toMap: Map[String, Any] = $resultMap"""
 83             val newTemplate = template"{ ..$stats } with ..$ctorcalls { $param => ..$newBody }"
 84 
 85             q"..$mods class $tname[..$tparams] (...$paramss) extends $newTemplate"
 86           }
 87         }
 88       case _ => throw new Exception("@Mappable can be annotation of class only")
 89     }
 90   }
 91 }
 92 import scala.util.Try
 93 @compileTimeOnly("@RetryOnFailure not expanded")
 94 class RetryOnFailure(repeat: Int) extends scala.annotation.StaticAnnotation {
 95   inline def apply(defn: Any): Any = meta {
 96     defn match {
 97       case q"..$mods def $name[..$tparams](...$paramss): $tpeopt = $expr" => {
 98         val q"new $_(${arg})" = this
 99         val repeats = Try(arg.toString.toInt).getOrElse(abort(s"Retry on failure takes number as parameter"))
100 
101         val newCode =
102           q"""..$mods def $name[..$tparams](...$paramss): $tpeopt = {
103                 import scala.util.Try
104 
105                 for( a <- 1 to $repeats){
106                   val res = Try($expr)
107                   if(res.isSuccess){
108                     return res.get
109                   }
110                 }
111 
112                 throw new Exception("Method fails after "+$repeats + " repeats")
113               }
114             """
115         newCode
116       }
117       case _ => abort("@RetryOnFailure can be annotation of method only")
118     }
119   }
120 }
121 
122 class TalkingAnimal(voice: String) extends StaticAnnotation {
123   inline def apply(defn: Any): Any = meta {
124     defn match {
125       case q"..$mods class $tname[..$tparams] (...$paramss) extends $template" =>
126         template match {
127           case template"{ ..$stats } with ..$ctorcalls { $param => ..$body }" => {
128             val q"new $_(${arg})" = this
129             val sound = arg.toString()
130             val animalType = tname.toString()
131             val newBody = body :+
132               q""" def sayHello: Unit =
133                      println("Hello, I‘m a " + $animalType +
134                     " and my name is " + name + " " + $sound+ "...")
135               """
136             val newTemplate =template"{ ..$stats } with ..$ctorcalls { $param => ..$newBody }"
137             q"..$mods class $tname[..$tparams] (...$paramss) extends $newTemplate"
138           }
139         }
140       case _ => abort("Error: expanding TalkingAnimal!")
141     }
142   }
143 }

试运行代码：

 1 object MacroAnnotDemo extends App {
 2   @Greetings object Greet {
 3     def add(x: Int, y: Int) = println(x + y)
 4   }
 5   @Greetings object Hi extends AnyRef {}
 6 
 7   Greet.sayHello("John")
 8   Greet.add(1,2)
 9   Hi.sayHello("Susana Wang")
10 
11   @Benchmark
12   def calcPow(x: Double, y: Double) = {
13     val z = x + y
14     math.pow(z,z)
15   }
16 
17   println(calcPow(4.2, 8.9))
18 
19   @Mappable
20   case class Car(color: String, model: String, year: Int, owner: String){
21     def turnOnRadio = {
22       "playing"
23     }
24   }
25 
26   val newCarMap = Car("Silver", "Ford", 1998, "John Doe").toMap
27   println(newCarMap)
28 
29   object utils {
30     def methodThrowingException(random: Int): Unit = {
31       if(random%2 == 0){
32         throw new Exception(s"throwing exception for ${random}")
33       }
34     }
35   }
36   import scala.util.Random
37   @RetryOnFailure(20) def failMethod[String](): Unit = {
38     val random = Random.nextInt(10)
39     println("Retrying...")
40     utils.methodThrowingException(random)
41   }
42 
43   trait Animal {
44     val name: String
45   }
46   @TalkingAnimal("wangwang")
47   case class Dog(val name: String) extends Animal
48 
49   @TalkingAnimal("miaomiao")
50   case class Cat(val name: String) extends Animal
51 
52   //@TalkingAnimal("")
53   //case class Carrot(val name: String)
54   //Error:(12,2) Annotation TalkingAnimal only apply to Animal inherited! @TalingAnimal
55   Dog("Goldy").sayHello
56   Cat("Kitty").sayHello
57 
58 }