project re-indexed every time opened #327
Comments
|
Hey @jackieli-tes The re-indexing is done on file changes. But it should only be re-indexing symbols in a file you've changed. This is usually done on |
|
Here it is: kotlin-langserver.log.zip It looks like only a couple requests were sent. A workspace didOpen, and a file didOpen. |
|
Thanks! That output helps. The language server logs confused me a bit, but the increase in number of symbols apparently makes sense (it's logging the total number of symbols indexed at that point in time, which includes previously indexed symbols as well). So the logs actually make sense to me now. Before #323 we indexed everything on startup, but nothing was done afterwards. This PR introduced indexing on file changes, but in order to support that, we basically ended up having to compile the entire project on startup (which was probably a good idea nonetheless). Based on the logs, I'm guessing the compilation of all the files is what's hurting your CPU. You'll notice the indexing of symbols takes only a couple of ms (per file) but the timestamps of the log messages suggest that compiling the files takes much longer. Taking this into account (and considering that the only real change on startup introduced by this PR is the compilation of all files) I'm guessing the compilation of all the files is what's overloading your CPU. I'm not sure how we could make this more performant right now though. I would have to look into it when I have more time. Do you mind sharing your specs and OS? I'm not even sure how to test this out, since I probably won't have a setup close to yours. Bear in mind that you're the first one to run into this problem (although to be fair, this change is not part of a release yet, so very few people probably tested this). |
|
Thanks for the details. If I understand correctly, the compilation is needed once and only if it's not compiled before? And compiled class & other files will be saved in My assumption is if I have opened the project once and wait for everything to be compliled, then I immediately close it and open again, it shouldn't need to compile it again? My setup is pretty straight forward:
My laptop spec (using |
Well, during a session this is true, but on startup it's not because the language server doesn't cache anything (I was actually about to suggest we implement this as a workaround instead).
As far as I know, we don't do any code generation.
This should be the way to go, but it's currently not implemented like that. I don't know if we have an issue to implement caching, but I think it would virtually solve your problem (and it would be a great feature to have). Thanks for the setup and specs. I'll try to find time to dig a bit more into it, but I can't promise I'll do it soon, since I'm quite busy. Introducing a language server cache looks much more promising though. |
|
Thanks for the explanation. However, I'm not sure I fully follow the "index" part.
My project is based on gradle and the compilation process would put all the class files under So you're suggestion the I can understand that the lsp server needs to load the symbols and definitions in memory, but the full source code compilation shouldn't be necessary right? |
|
Yeah, perhaps I wasn't clear. Let me try to explain it better. When I'm talking about the language server compiling the files, I'm not talking about the usual compilation you do when you invoke gradle. We use the Kotlin compiler API to compile files. Whether the compiler API itself does things in memory or using files, I don't know, since it's a vast code base. But the bottom line is that we don't generate code (i.e., it's not a full compilation). Note that we still need to compile (perhaps "analyze" makes more sense to you here) the files in order to check for errors, so that's what we're doing here. |
|
I see. Now they all make sense. Thanks for the explanation! I don't see any issue logged for the "index cache". IMHO it's definitely needed: my project is not very big: about 43K LOC, and now every time I open it, the system feels sluggish for a few minutes. |
|
#160 touches a bit on caching dependencies but yeah, I don't think there's a dedicated issue for caching. Note that I'm talking about caching multiple things. The symbol index would be a good thing to cache, especially since indexing dependencies can take a while, but the one that will solve your problem is caching the compilations of the files i.e., the IR (assuming the compilation of the files is what's overloading the CPU, that is). I created a dedicated issue for it: #328 |
Hi,
Firstly, thanks for this awesome project.
My setup is Neovim 0.6.1 with coc.nvim. I'm staying on the HEAD of the master branch. Since #323 , I'm experience huge CPU usage every time I'm opening a project. I can see in the log it's trying to "update symbol index", but I'm trying to figure out what I'm doing that triggers the "re-indexing", seemingly to the whole project every time I open it. With vim, it's more obvious because I open and close it quite often.
the language server log from coc
The text was updated successfully, but these errors were encountered: