Do not crash in backend when method is too large #14213
Comments
|
you have duplicated your calls to inline def getTypeclassInstances[F[_], A <: Tuple]: List[F[Any]] =
inline erasedValue[A] match {
case _: EmptyTuple => Nil
case _: (head *: tail) =>
val headTypeClass =
summonInline[F[head]]
val tailTypeClasses =
getTypeclassInstances[F, tail]
- headTypeClass.asInstanceOf[F[Any]] :: getTypeclassInstances[F, tail]
+ headTypeClass.asInstanceOf[F[Any]] :: tailTypeClasses
}in general though this could still crash with a very large case class - perhaps you can change |
|
Oh, Good advice ! Thank you, it does help me. |
bishabosha
changed the title
|
I say we could reopen this to give a better error message to the user, rather than crash |
bishabosha
added
itype:crash
itype:bug
Spree
bishabosha
changed the title
|
I have found a related example (below, minimised from a real word application) that results in an “java.lang.OutOfMemoryError: Java heap space” on a 24GB heap. Only a slight modification results in an "MethodTooLargeException". So probably there is more going on behind the scenes. Previously discussed on the Scala Users Forum. /** Compile with:
sbt.version=1.6.2
scalaVersion := "3.1.1"
scalacOptions ++= Seq("-feature","-deprecation","-unchecked","-explain"),
libraryDependencies ++= Seq("com.softwaremill.quicklens" %% "quicklens" % "1.8.3")
*/
package HeapError
import com.softwaremill.quicklens._
/* Presence of AnyVal turns 'MethodTooLargeException' into an
'java.lang.OutOfMemoryError: Java heap space' error */
class Moment(val value: Long) extends AnyVal with Ordered[Moment]
{ def compare(that: Moment) = (this.value - that.value).sign.toInt
def isZero = value == 0 }
case class Timers(
wakeTime: Moment,
constructionTime: Moment,
initializationTime: Moment,
activationTime: Moment,
clock: Moment,
lastSignal: Moment,
lastTransmission: Moment,
lastHeartbeat: Moment,
lastClockSet: Moment,
lastFail: Moment)
case class Counters(
missedAcks: Int,
upSinceHB: Int,
downSinceHB: Int,
failSinceHB: Int,
upSum: Int,
downSum: Int,
failSum: Int)
case class SystemState(
mcuTemperature: Double,
batteryCharge: Double,
bufferSpace: Double,
health: Double)
case class Monitor(timers: Timers, counters: Counters, systemState: SystemState)
{ def setClockInit(time: Moment): Monitor = this
.modifyAll(_.timers.clock,
_.timers.initializationTime,
_.timers.lastClockSet,
_.timers.lastSignal,
_.timers.lastTransmission)
.setToIf(!time.isZero)(time) } |
|
Minimized original to: inline def g(inline x: Int): List[Int] =
inline if x == 0 then Nil else
val headTypeClass = 1
val tailTypeClasses = g(x - 1)
headTypeClass.asInstanceOf[Int] :: g(x - 1)
def f = g(11)Exception in thread "main" scala.tools.asm.MethodTooLargeException: Method too large: Test$package$.f ()Lscala/collection/immutable/List; while compiling /Users/kmetiuk/Projects/scala3/playground/learning/Test.scala
scala.tools.asm.MethodTooLargeException: Method too large: Test$package$.f ()Lscala/collection/immutable/List;
at scala.tools.asm.MethodWriter.computeMethodInfoSize(MethodWriter.java:2087)
at scala.tools.asm.ClassWriter.toByteArray(ClassWriter.java:489)
at dotty.tools.backend.jvm.GenBCodePipeline$Worker2.getByteArray$1(GenBCode.scala:484)
at dotty.tools.backend.jvm.GenBCodePipeline$Worker2.addToQ3(GenBCode.scala:490)
at dotty.tools.backend.jvm.GenBCodePipeline$Worker2.run(GenBCode.scala:467)
at dotty.tools.backend.jvm.GenBCodePipeline.buildAndSendToDisk(GenBCode.scala:568)
at dotty.tools.backend.jvm.GenBCodePipeline.run(GenBCode.scala:531)
at dotty.tools.backend.jvm.GenBCode.run(GenBCode.scala:67)
at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:311)
at scala.collection.immutable.List.map(List.scala:246)
at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:312)
at dotty.tools.backend.jvm.GenBCode.runOn(GenBCode.scala:75)
at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:225)
at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1328)
at dotty.tools.dotc.Run.runPhases$1(Run.scala:236)
at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:244)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
at dotty.tools.dotc.Run.compileUnits(Run.scala:253)
at dotty.tools.dotc.Run.compileSources(Run.scala:186)
at dotty.tools.dotc.Run.compile(Run.scala:170)
at dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
at dotty.tools.dotc.Driver.process(Driver.scala:195)
at dotty.tools.dotc.Driver.process(Driver.scala:163)
at dotty.tools.dotc.Driver.process(Driver.scala:175)
at dotty.tools.dotc.Driver.main(Driver.scala:205)
at dotty.tools.dotc.Main.main(Main.scala)This issue was picked for the Issue Spree 14 of 12th April. @dos65 and @anatoliykmetyuk will be working on it. If you have any insight into the issue or guidance on how to fix it, please leave it here. |
|
What fix do you have in mind? A better error message when inlining leads to a method that's too large? Estimating the size of a method from a compiler AST seems difficult, but maybe a warning once a certain threshold is passed would work? Even with an ASM |
|
The crash in question seems to be caused by an exponential space complexity of code generated by the Inliner. It is caused by the user input to the Inliner: each call to
I am in favor of Inliner-time error reporting for the following reasons:
See also #9203 and #9237 – not merged since I didn't have the time to work on the requested adjustments; is not too related to this issue anyway. But maybe we want some kind of generic space complexity tracking solution shared between all compile-time computations. The problem in its general form: Scala's compiler is capable of running compile-time computations that, under certain user input, will generate trees that takes too much space. WDYT @nicolasstucki ? |
|
During the spree, we discussed this issue with @nicolasstucki and decided that there aren't any good approach to properly detect that tree is too large at the Inliner phase. @anatoliykmetyuk should we close this one? |
|
Yes, this can be closed once #14943 is merged. |
When I use Mirror of scala 3 to generate a typeclass list, the exception occurs. I know it's the hard limit of jvm of method size, but how can I circumvent this issue.
ps: When delete some fields of Data class it works, but any other solution?
Compiler version
sbt: 1.6.0
scala: 3.1.0
Minimized example
Output
The text was updated successfully, but these errors were encountered: