Some of my Scala files have lines marked as errors despite being valid Scala code

[scabug]

To reproduce, please install Scala plugin and subclipse plugin and import the source code from https://asn1gen.googlecode.com/svn/trunk/asn1gen as an eclipse project.

Then open the file asn1gen/src/org.asn1gen.parsing.asn1.Asn1ParserBase.scala

Here, I get a lot of "not found" and "not a member" errors.

This is despite the fact that whole project builds fine and the corresponding unit tests at https://asn1gen.googlecode.com/svn/trunk/asn1gentest all pass.

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-2767?orig=1
Reporter: John Ky (newhoggy)
Attachments:

• ScalaFail.zip (created on Jan 20, 2010 3:45:49 PM UTC, 8224 bytes)
• Screen%20shot%202010-01-24%20at%201.00.00%20PM.png (created on Jan 24, 2010 1:04:53 AM UTC, 73464 bytes)
• Screen%20shot%202010-01-24%20at%201.09.39%20PM.png (created on Jan 24, 2010 1:10:29 AM UTC, 93264 bytes)

[scabug]

spiros said:
Which version of the plug-in are you using?

If you are using 2.7.x, is this bug the same as SI-1277?

[scabug]

John Ky (newhoggy) said:
I was using a nightly build downloaded about the same time I made this ticket.

[scabug]

John Ky (newhoggy) said:
Scala Eclipse Plugin 2.8.0.r19928-b20091129034731 ch.epfl.lamp.sdt.feature.group

[scabug]

spiros said:
I just did a svn checkout of you project and I see no errors in the file Asn1ParserBase.scala. Do those errors appear after a Clean and auto-build? If this is the case then this is probably a duplicate of #2689.

BTW thank you for you interest in improving SDT by submitting a bug report. Bug reports are always helpful, but they are more helpful if they provide a specific way to reproduce the bug using a small self-contained code example. For example see #2689, the code provided is minimal and has no dependencies so the developers can easily see the problem or use the provided code in a test case.

It would be extremely helpful, if this bug is not a duplicate of #2689, if you could find the smallest example that reproduces the misbehavior that you have observed.

[scabug]

John Ky (newhoggy) said:
The errors go away if I simplify it too much.

I've got a screencast showing the errors I am getting with a slightly more simplified class, but I think it must be part of the project.

http://www.screencast.com/users/newhoggy/folders/Jing/media/cc7e1259-b0af-4175-9e5b-857a0a3d53f0

Let me know if there is anything else I can provide to help diagnose the problem.

[scabug]

spiros said:
If I understood correctly the errors appear after a Clean operation. This is probably related to [http://old.nabble.com/Re%3A--scala-tools--Error-highlights-in-Eclipse-Scala-p26480821.html this] behavior.

BTW when you are using the nightly builds you don't have to report the whole version number, you can just use the revision. In you case it is r19928.

[scabug]

John Ky (newhoggy) said:
Yes, it does appear after a clean operation. I didn't notice it had to do with the clean operation because clean & build is how I've been working for a while now. I find clean & build is more stable and avoid the automatic build because the atomic build occasionally gets into a 100% cpu state and wastes away my battery while I'm working on the train.

[scabug]

@milessabin said:
I'd be very grateful if someone could review this ticket against the current state of trunk.

[scabug]

John Ky (newhoggy) said:
I must say, you did an excellent job of it. I don't see any erroneous errors anymore.

[scabug]

spiros said:
Unfortunately I still see bogus error messages when I do a Clean&Build in some of my projects.

[scabug]

spiros said:
BTW I am using r20428.

[scabug]

John Ky (newhoggy) said:
You're right. I still get some, but way fewer than before.

For some reason, it doesn't like:

case NULL => {

NULL and ValueReference.

case lexical.Operator(chars) => Operator(chars)

case lexical.CommentLit(n) => n

I'm using r20423

[scabug]

@milessabin said:
Small self-contained examples would be very helpful (though I would expect it will take more than one source file to exhibit any problems).

[scabug]

spiros said:
See also: #2889.

[scabug]

@milessabin said:
#2889 is now fixed, so I'd be grateful for another review of this ticket ... unfortunately I'm not expecting this one to be fixed.

[scabug]

spiros said:
Hi Miles,

any compiler related fix is more than welcome, I am looking forward to test the next nightly.

Replying to [comment:15 milessabin]:

#2889 is now fixed, so I'd be grateful for another review of this ticket ... unfortunately I'm not expecting this one to be fixed.

[scabug]

spiros said:
I tried to find a small example that reproduces this bug but unfortunately I cannot reproduce it in a small project. This bug is still valid (using r20518) but appears less frequently.

[scabug]

@milessabin said:
Well, I guess that's progress of a sort. Thanks for your efforts, and if you do come up with any more reproducible examples they'd be extremely welcome.

[scabug]

spiros said:
Hi Miles,

the below code gives a bogus error message in the Scala editor:

package test

import javax.persistence._

class User {

  @Temporal(TemporalType.TIMESTAMP)
  var lastLogin: java.util.Date = _

}

The message is: "annotation argument needs to be a constant; found: TemporalType{}.TIMESTAMP{}" and I believe that it happens because the Scala compiler believes that Java Enums are not valid values for annotation arguments (TemportalType is a Java Enum).

Sorry for the external dependency to JPA but this does not work if the annotation and the enum are defined in the same project as the User class (maybe due to a compiler bug, I'll file another bug report).

[scabug]

@milessabin said:
Unless I'm very much mistaken, that example is a dupe of #2764 which is actually a compiler bug.

[scabug]

spiros said:
Thanks for the heads up! I was just writing a duplicate bug report of #2764.

If the Java annotation, Java enum and the Scala class are in the same project then we get the compiler bug and Eclipse correctly reports this error in both the editor and the problems view (Eclipse does not know that the compiler has a bug).

In the case that I wrote in my comment the code compiles and runs so the error is bogus since compilation completed successfully. The problem with Eclipse here is that an error that produced during compilation and was later (during compilation) removed is shown in the Scala editor.

[scabug]

spiros said:
Your [http://old.nabble.com/Re%3A--scala-tools--Error-highlights-in-Eclipse-Scala-p26480821.html message] in the scala-tools mailing list is my guide for researching this issue. You said:

...or where early compilation failures have prevented otherwise good code
from being built which then has knock on effects on code which depends on
it (sounds like what you're experiencing).

and I believe that since the code gets compiled this might be an instance of this behavior.

[scabug]

spiros said:
From #2764 "it works when compiling against the compiled java files" which is the case in my submitted example (since the annotation and the enum are in the JPA jar).

[scabug]

@milessabin said:
Do you have a source attachment for the JPA jar? If you do, could you remove it and then try reproducing the bogus error.

[scabug]

spiros said:
I can reproduce this in a Maven project with the JPA sources attached and in a plain Scala project without the sources attached. There is no difference. Do you have any problem reproducing the error?

[scabug]

@milessabin said:
I think I'm still not fully understanding your comment 21. How does the IDE's behaviour differ from what you'd expect, given that #2764 isn't yet fixed?

[scabug]

spiros said:
The code gets compiled so I do not expect to see any error messages. Also the error message appears in the Scala editor but not in the Problems View (which is a sign that the error message is bogus).

[scabug]

spiros said:
On bug #2764

The creator of the bug said that the attached Scala file (which is the same as the example that I used in my comment) produces a compilation error but the code runs (so it compiles). I believe that he was seeing a bogus message (not reported in the Problems View). The message must be bogus since the he said that the code runs. If we get a compilation error we cannot run the code.

Then when you tried to minimize the dependencies of the example code you found the real bug with the code that you wrote in your first comment. In this comment the annotation, enum and Scala class are defined (and hence compiled) in the same project so the compiler bug get triggered (Eclipse here correctly reports the compilation error which is result of the bug since it does not know that the compiler has a bug). In this case you cannot run the code and hence the compilation error in valid (although the result of a compiler bug).

I expect the code that uses the JPA annotations not to produce any errors because as rytz said in a later comment of #2764 "it works when compiling against the compiled java files" which is the case with JPA since the annotation and the enum are compiled in the JPA jar. Also when we compile from the command line we do not get any compiler messages.

[scabug]

@milessabin said:
Understood. But I still don't see the Eclipse issue here: the difference in behaviour is due to the presentation compiler using source if available whereas the build compiler only ever uses the binaries and the compiler bug only affecting the source path. I think that if the compiler bug is fixed the issue in this particular example goes away.

The foregoing is assuming that you do have source in this case ... hence my earlier question about source attachments.

[scabug]

@milessabin said:
Thanks, but that doesn't answer my question ;-)

I suspect that the fix for #2931 will have improved the situation for this ticket as well, but there are still likely to be issues in the presentation compiler with types which are contained in source files which aren't named for the type.

If automatic building is enabled, then the class file generated for such a type will provide a reference back to the source file it was built from irrespective of the source file name. But if there's no class file there's no way (currently) for the presentation compiler to locate the source file for a given type (unless it already knows about it because that type's source file is open in an editor).

Hopefully that gives you some context for my previous request ...

[scabug]

spiros said:
I always have automatic building enabled.

[scabug]

@milessabin said:
OK, thanks. If you could let me know if r20660 has improved things for you that would be very helpful.

[scabug]

spiros said:
Replying to [comment:46 milessabin]:

OK, thanks. If you could let me know if r20660 has improved things for you that would be very helpful.

Of course I will!

I just hope that the nightly build won't be broken again so I can update tomorrow morning (last success was 2 days 17 hr ago).

[scabug]

@milessabin said:
There shouldn't be anything wrong with the nightlies. Do you run Eclipse with the -clean option? Either way, try uninstalling and reinstalling.

[scabug]

@ijuma said:
There is a stability error (see scala-internals) and that's why we haven't had nightlies for the last few days.

[scabug]

spiros said:
Replying to [comment:49 ijuma]:

There is a stability error (see scala-internals) and that's why we haven't had nightlies for the last few days.

Yeap! If I am correct there are some scaladoc errors that prevent the build to succeed so the update site does not get updated.

[scabug]

@ijuma said:
I am not aware of scaladoc problems, but the stability issue has been fixed by Martin just now (by "stability", I the test.stability ant target).

[scabug]

spiros said:
Replying to [comment:51 ijuma]:

I am not aware of scaladoc problems, but the stability issue has been fixed by Martin just now (by "stability", I the test.stability ant target).

Sorry my mistake :-(

I quickly saw the console output from hudson and thought it was a scaladoc problem. Good to know that the problem is fixed.

[scabug]

spiros said:
I've tested this with r20669 and unfortunately the bug is still valid. So far I've seen exactly the same behavior.

[scabug]

@milessabin said:
I now have a minimal reproducer ...

A.scala

object Misnamed

B.scala

object B {
  val a = Misnamed
} 

If at least one of,

• A.scala is open in an editor
  or,
• There is a class file for object Misnamed
  Then the editor will (correctly) not report errors against the occurrence of Misnamed in B.scala. If both of these conditions do not hold (ie. close A.scala, turn off automatic building, then do a clean without an immediate rebuild) then the editor will (incorrectly) report "not found: value Misnamed" against the occurrence of Misnamed in B.scala.

Note that in the both the error and the non-error case the presentation compiler might need a little nudge to exhibit the behaviour as described (e.g to make the incorrect error go away by opening A.scala you might also need to make a single character edit to B.scala; to make the error reappear you might need to reinstantiate the presentation compiler by doing a clean).

I think that this example probably covers all the remaining cases of this ticket, so if you're seeing incorrect behaviour which is clearly different from the incorrect behaviour I've just described, then please let me know.

[scabug]

spiros said:
I see a different behavior.

I have a small 58 file project. In that project almost all of the classes are placed in files that correspond to their class names except for one exception which is placed in a file called exceptions.scala. In that project when I do a Clean&Build I get the bogus errors despite exceptions.scala being open and .class file being present in the hard disk. Closing the file and reopening it does not remove the errors. To remove them I have to edit the file and save it.

From my testing so far, what appears to have changed in the recent nightly is that we don't longer get bogus error messages for incremental compilation. Now to trigger the bug we need to do a clean.

BTW I can privately share two projects that have this behavior. They are 25 and 58 files and they only depend on JUnit. I've tried to trimmed them down to a 5 files project but the errors disappear. If you want I can send them via email.

[scabug]

@milessabin said:
Yes, if you could send me the smaller of the two projects that would be very helpful.

[scabug]

spiros said:
BTW the bug #2710 might be related to this bug.

[scabug]

@milessabin said:
I must be missing some connection that you see ... why do you think they're related?

[scabug]

spiros said:
You are right, I didn't mean to imply that there in a common underlining cause for the two bugs. My reasoning is that since they both happen after a clean and since (if I am correct) the task items are added by the presentation compiler there might be various inconsistencies that happen in the state of the presentation compiler after a clean operation.

Related in my previous comment meant that the fix for #2710 and the fix this bug might be in the same area so #2710 might be a good bug to tackle after this one.

[scabug]

@milessabin said:
Actually, task items are added by the build compiler.

[scabug]

spiros said:
I am obviously wrong, sorry for the noise.

[scabug]

@milessabin said:
I finally have a fix for this ... a little tidying up to do before I commit, but it'll be going in tomorrow.

[scabug]

@milessabin said:
Fixed in r20835 and r20839.

The IDE now maintains a map from top-level symbols to source files, and uses this to add additional compilation units to the presentation compiler run to resolve symbols which can't be resolved via the Java source naming convention.

Building the map requires parsing the sources to extract top-level symbols. This adds a (small) one-time overhead the first time a project is touched after a restart or a clean, and a (very small) overhead on each source file save. I think this overhead is acceptable (4s for the initial whole-project scan for the scala project, 10s of ms for single file scans), but if that turns out not to be the case it should be possible to persist some or all of the map across restarts and cleans.

I believe that this fixes all of the issues mentioned in this ticket so far. However, there might be further cases where the presentation compiler reports spurious errors. Please use your judgement when deciding whether to reopen this ticket or create a fresh one: if you have a very specific, easily reproducible example, then a fresh ticket would be appropriate.

[scabug]

spiros said:
I am using r20847 and I still see spurious errors :-(

From my testing so far the errors are greatly reduced but they still exist. At first I thought that the errors were only for packages and package objects but I also saw an error for an object. I will investigate this further and try to report specific ways to reproduce.

All the errors are for Scala top level elements that are defined in files that do not correspond to their qualified names that is why I reopen this issue instead of a new one.

[scabug]

@milessabin said:
Have you had any luck finding new reproducible examples of this? I'm not seeing anything ...

[scabug]

spiros said:
I sent you via email instructions on how to reproduce this error using the second project that I've emailed you in past. Have you tried using those instructions?

I am using r20879 and I definitely see spurious errors. If you cannot reproduce this bug using the instructions in my email please let me know and I'll try to come up with another example.

[scabug]

@milessabin said:
Fixed in r20968 (hopefully).