Skip to content

Cache context base base in quote driver #4643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 27, 2018
Merged

Cache context base base in quote driver #4643

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8ca5af1
Cache compilere driver in quote.Toolbox
nicolasstucki Jun 10, 2018
2535c2d
Revert "Disable tests that timeout easily"
nicolasstucki Jun 19, 2018
f781701
Add cached toolbox to test
nicolasstucki Jun 19, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions compiler/src/dotty/tools/dotc/quoted/QuoteDriver.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,21 @@ package dotty.tools.dotc.quoted

import dotty.tools.dotc.ast.tpd
import dotty.tools.dotc.Driver
import dotty.tools.dotc.core.Contexts.Context
import dotty.tools.dotc.core.Contexts.{Context, ContextBase}
import dotty.tools.io.{AbstractFile, Directory, PlainDirectory, VirtualDirectory}
import dotty.tools.repl.AbstractFileClassLoader

import scala.quoted.{Expr, Type}
import java.net.URLClassLoader

import Toolbox.{Run, Settings, Show}
import dotty.tools.dotc.tastyreflect.TastyImpl

class QuoteDriver extends Driver {
import tpd._

def run[T](expr: Expr[T], settings: Settings[Run]): T = {
private[this] val contextBase: ContextBase = new ContextBase
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is thread-safe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not and not intended to be. Each thread should generate its own toolbox. I will add documentation and thick how to check that they do not use the same compiler in two threads.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided to make it thread-safe by synchronization instead. If someone wants parallelism they should create several instances of the Toolbox

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided to make it thread-safe by synchronization instead.

I don't see any synchronization logic in the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must have lost that commit in one of the reading. I will add it again.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Searching for synchronized still finds nothing here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is in #4746


def run[T](expr: Expr[T], settings: ToolboxSettings): T = {
val (_, ctx: Context) = setup(settings.compilerArgs.toArray :+ "dummy.scala", initCtx.fresh)

val outDir: AbstractFile = settings.outDir match {
Expand All @@ -39,15 +40,15 @@ class QuoteDriver extends Driver {
method.invoke(instance).asInstanceOf[T]
}

def show(expr: Expr[_], settings: Settings[Show]): String = {
def show(expr: Expr[_], settings: ToolboxSettings): String = {
def show(tree: Tree, ctx: Context): String = {
val tree1 = if (settings.rawTree) tree else (new TreeCleaner).transform(tree)(ctx)
TastyImpl.showSourceCode.showTree(tree1)(ctx)
}
withTree(expr, show, settings)
}

def withTree[T](expr: Expr[_], f: (Tree, Context) => T, settings: Settings[_]): T = {
def withTree[T](expr: Expr[_], f: (Tree, Context) => T, settings: ToolboxSettings): T = {
val (_, ctx: Context) = setup(settings.compilerArgs.toArray :+ "dummy.scala", initCtx.fresh)

var output: Option[T] = None
Expand All @@ -59,7 +60,7 @@ class QuoteDriver extends Driver {
output.getOrElse(throw new Exception("Could not extract " + expr))
}

def withTypeTree[T](tpe: Type[_], f: (TypTree, Context) => T, settings: Settings[_]): T = {
def withTypeTree[T](tpe: Type[_], f: (TypTree, Context) => T, settings: ToolboxSettings): T = {
val (_, ctx: Context) = setup(settings.compilerArgs.toArray :+ "dummy.scala", initCtx.fresh)

var output: Option[T] = None
Expand All @@ -72,7 +73,7 @@ class QuoteDriver extends Driver {
}

override def initCtx: Context = {
val ictx = super.initCtx.fresh
val ictx = contextBase.initialCtx
var classpath = System.getProperty("java.class.path")
this.getClass.getClassLoader match {
case cl: URLClassLoader =>
Expand Down
52 changes: 5 additions & 47 deletions compiler/src/dotty/tools/dotc/quoted/Toolbox.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,67 +4,25 @@ import dotty.tools.dotc.ast.tpd

import scala.quoted.Expr
import scala.quoted.Exprs.{LiftedExpr, TastyTreeExpr}
import scala.runtime.quoted._

/** Default runners for quoted expressions */
object Toolbox {
import tpd._

type Run
type Show
implicit def make(implicit settings: ToolboxSettings): scala.quoted.Toolbox = new scala.quoted.Toolbox {

implicit def toolbox[T](implicit
runSettings: Settings[Run] = Settings.run(),
showSettings: Settings[Show] = Settings.show()
): Toolbox[T] = new Toolbox[T] {
private[this] val driver: QuoteDriver = new QuoteDriver()

def run(expr: Expr[T]): T = expr match {
def run[T](expr: Expr[T]): T = expr match {
case expr: LiftedExpr[T] =>
expr.value
case expr: TastyTreeExpr[Tree] @unchecked =>
throw new Exception("Cannot call `Expr.run` on an `Expr` that comes from an inline macro argument.")
case _ =>
new QuoteDriver().run(expr, runSettings)
driver.run(expr, settings)
}

def show(expr: Expr[T]): String = new QuoteDriver().show(expr, showSettings)
def show[T](expr: Expr[T]): String = driver.show(expr, settings)

}

class Settings[T] private (val outDir: Option[String], val rawTree: Boolean, val compilerArgs: List[String])

object Settings {

/** Quote run settings
* @param optimise Enable optimisation when compiling the quoted code
* @param outDir Output directory for the compiled quote. If set to None the output will be in memory
* @param compilerArgs Compiler arguments. Use only if you know what you are doing.
*/
def run(
optimise: Boolean = false,
outDir: Option[String] = None,
compilerArgs: List[String] = Nil
): Settings[Run] = {
var compilerArgs1 = compilerArgs
if (optimise) compilerArgs1 = "-optimise" :: compilerArgs1
new Settings(outDir, false, compilerArgs1)
}

/** Quote show settings
* @param color Print output with colors
* @param rawTree Do not remove quote tree artifacts
* @param compilerArgs Compiler arguments. Use only if you know what you are doing.
*/
def show(
color: Boolean = false,
rawTree: Boolean = false,
compilerArgs: List[String] = Nil
): Settings[Show] = {
var compilerArgs1 = compilerArgs
compilerArgs1 = s"-color:${if (color) "always" else "never"}" :: compilerArgs1
new Settings(None, rawTree, compilerArgs1)
}

}

}
28 changes: 28 additions & 0 deletions compiler/src/dotty/tools/dotc/quoted/ToolboxSettings.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package dotty.tools.dotc.quoted

class ToolboxSettings private (val outDir: Option[String], val rawTree: Boolean, val compilerArgs: List[String])

object ToolboxSettings {

implicit def default: ToolboxSettings = make()

/** Make toolbox settings
* @param optimise Enable optimisation when compiling the quoted code
* @param outDir Output directory for the compiled quote. If set to None the output will be in memory
* @param color Print output with colors
* @param rawTree Do not remove quote tree artifacts
* @param compilerArgs Compiler arguments. Use only if you know what you are doing.
*/
def make(
optimise: Boolean = false,
color: Boolean = false,
rawTree: Boolean = false,
outDir: Option[String] = None,
compilerArgs: List[String] = Nil
): ToolboxSettings = {
var compilerArgs1 = compilerArgs
if (optimise) compilerArgs1 = "-optimise" :: compilerArgs1
new ToolboxSettings(outDir, rawTree, compilerArgs1)
}

}
5 changes: 2 additions & 3 deletions library/src/scala/quoted/Expr.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package scala.quoted

import scala.runtime.quoted.Toolbox
import scala.runtime.quoted.Unpickler.Pickled

sealed abstract class Expr[T] {
Expand All @@ -10,10 +9,10 @@ sealed abstract class Expr[T] {
*
* May throw a FreeVariableError on expressions that came from an inline macro.
*/
final def run(implicit toolbox: Toolbox[T]): T = toolbox.run(this)
final def run(implicit toolbox: Toolbox): T = toolbox.run(this)

/** Show a source code like representation of this expression */
final def show(implicit toolbox: Toolbox[T]): String = toolbox.show(this)
final def show(implicit toolbox: Toolbox): String = toolbox.show(this)
}

object Expr {
Expand Down
9 changes: 9 additions & 0 deletions library/src/scala/quoted/Toolbox.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package scala.quoted

import scala.annotation.implicitNotFound

@implicitNotFound("Could not find implicit quoted.Toolbox.\n\nDefault toolbox can be instantiated with:\n `implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make`\n\nIf only needed once it can also be imported with:\n `import dotty.tools.dotc.quoted.Toolbox._`")
trait Toolbox {
def run[T](expr: Expr[T]): T
def show[T](expr: Expr[T]): String
}
10 changes: 0 additions & 10 deletions library/src/scala/runtime/quoted/Toolbox.scala
View file
Open in desktop

This file was deleted.

15 changes: 15 additions & 0 deletions tests/run-with-compiler-custom-args/staged-streams_1.check
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,18 @@

12

36

2

3

7

12

15

15

72
34 changes: 17 additions & 17 deletions tests/run-with-compiler-custom-args/staged-streams_1.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
import dotty.tools.dotc.quoted.Toolbox._
import scala.quoted._

/**
Expand Down Expand Up @@ -674,26 +673,27 @@ object Test {
.fold('{0}, ((a: Expr[Int], b : Expr[Int]) => '{ ~a + ~b }))

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

println(test1().run)
println
println(test2().run)
println
// FIXME re-enable in #4643 when we can cache the compiler context
// println(test3().run)
// println
// println(test4().run)
// println
// println(test5().run)
// println
// println(test6().run)
// println
// println(test7().run)
// println
// println(test8().run)
// println
// println(test9().run)
// println
// println(test10().run)
println(test3().run)
println
println(test4().run)
println
println(test5().run)
println
println(test6().run)
println
println(test7().run)
println
println(test8().run)
println
println(test9().run)
println
println(test10().run)
}
}

Expand Down
2 changes: 2 additions & 0 deletions tests/run-with-compiler/i3876-b.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import dotty.tools.dotc.quoted.Toolbox._
import scala.quoted._
object Test {
def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

val x: Expr[Int] = '(3)

val f2: Expr[Int => Int] = '{
Expand Down
2 changes: 2 additions & 0 deletions tests/run-with-compiler/i3876-c.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import dotty.tools.dotc.quoted.Toolbox._
import scala.quoted._
object Test {
def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

val x: Expr[Int] = '(3)

val f3: Expr[Int => Int] = '{
Expand Down
2 changes: 2 additions & 0 deletions tests/run-with-compiler/i3876-d.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import dotty.tools.dotc.quoted.Toolbox._
import scala.quoted._
object Test {
def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

val x: Expr[Int] = '(3)

val f4: Expr[Int => Int] = '{
Expand Down
2 changes: 2 additions & 0 deletions tests/run-with-compiler/i3876.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import dotty.tools.dotc.quoted.Toolbox._
import scala.quoted._
object Test {
def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

val x: Expr[Int] = '(3)

val f: Expr[Int => Int] = '{ (x: Int) => x + x }
Expand Down
2 changes: 2 additions & 0 deletions tests/run-with-compiler/i3946.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ import dotty.tools.dotc.quoted.Toolbox._
import scala.quoted._
object Test {
def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

val u: Expr[Unit] = '()
println(u.show)
println(u.run)
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947b.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947b2.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947b3.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947c.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947d.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947d2.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947e.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947f.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
1 change: 1 addition & 0 deletions tests/run-with-compiler/i3947g.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import dotty.tools.dotc.quoted.Toolbox._
object Test {

def main(args: Array[String]): Unit = {
implicit val toolbox: scala.quoted.Toolbox = dotty.tools.dotc.quoted.Toolbox.make

def test[T](clazz: java.lang.Class[T]): Unit = {
val lclazz = clazz.toExpr
Expand Down
Loading