[Multiplatform + Neovim + VSCode] Performance, unresolved references, errors

[avently]

Hello, thank you for the work on this server. Trying to make it work with Neovim. I'm using Intellij IDEA maaaany years and the project I testing now works in IDEA beautifully. So I suppose I need to somehow setup the server if I want to use it with Neovim.
I filling this issue with multiple problems because I think that all issues related to one thing that I'm missing. If it's not the case, tell me to create multiple small issues or to read docs:)

Prerequests:

• Neovim 0.7.0
• Plugin https://github.com/neovim/nvim-lspconfig with Kotlin config: https://github.com/neovim/nvim-lspconfig/blob/master/lua/lspconfig/server_configurations/kotlin_language_server.lua
• LSP installer in use: https://github.com/williamboman/nvim-lsp-installer/
• Archlinux x86_64
• 4 cores & 16 GB RAM (cloud)
• installed kotlin via pacman -S kotlin
• installed gradle with pacman -S gradle

Issues I have:

Overload resolution ambiguity

It's a main problem. The main thing for KMPP is not working properly. While Goto reference works perfectly within MY common/native/Android code. The server able to find reference to a function from different source set.

Unresolved references

Third-party libraries don't work. While all of them work in Intellj IDEA.

Low performance

After reopening vim (closing vim and reopening it again) on the same file I worked before it takes 4-8 minutes to see first error highlighting. All this time CPU almost on max. Memory usage increases from 1G to 4-5GB easily (numbers without including Gradle memory usage). Almost all CPU usage from the server, not Gradle.

When the server started (for the moment when I see highlighted errors), I try to edit something. So I just removing a piece of code to make the server showing an error. Error is not highlighted even after a couple of minutes. In IDEA it takes 2-3 seconds on a long file.

Screenshot after 3 minutes of loading the project:

Video about not highlighted error in a code after making an error:

kotlin-2022-07-29_23.39.32.mp4

Project structure:

.
├── android
│   ├── build.gradle
│   └── src
│       ├── main
│       └── test
├── build.gradle
├── common
│   ├── build.gradle
│   └── src
│       ├── androidMain
│       ├── commonMain
│       ├── desktopMain
│       └── jvmMain
├── core
│   ├── build.gradle
│   ├── gradle
│   │   └── wrapper
│   ├── gradle.properties
│   ├── gradlew
│   ├── gradlew.bat
│   └── src
│       ├── commonMain
│       ├── commonTest
│       ├── jvmMain
│       └── nativeMain
├── desktop
│   ├── build.gradle.kts
│   └── src
│       └── jvmMain
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── gradle.properties
├── gradlew
├── gradlew.bat
├── local.properties
└── settings.gradle

settings.gradle

rootProject.name = 'MyProject'
include ':core'
include ':common'
include ':android'
include ':desktop'

What am I doing wrong?

[avently]

Just checked, same problems applies to VSCode Kotlin extension with two differences: the extension reacts on a situation when I make an error. In this case it successfully shows an error after 3-4 seconds which is acceptable delay in my opinion. And startup time for the moment of displaying an error is about 30 seconds which is good.
Other problems are the same: conflicting overloads, overload resolution ambiguity, no external libs support.

[themkat]

no external libs support

That sounds extremely weird 😱 Are you sure the indexing dependencies operations are done at that point? (in VSCode you have a kotlin-option in the output section, and I usually check my *lsp-log*-buffer in Emacs. Unsure about neovim). In larger multi module projects, it can take quite a while for these operations to be done (example: detekt source code usually takes a few minutes before everything is loaded). In smaller projects with dependencies it is usually loaded in some seconds (10+). Never used it with Android, so unsure what the issue might be. The language server should support Maven and Gradle. Would be really useful to be able to see some outputs from the language server itself 🙂 If you are not able to find it, do you have a minimal project you can share where you also experience these issues? Then maybe I can investigate a bit.

Sorry if I seem surprised. In newer times I have not had any issues with external libs in Maven and Gradle 🙁

Issues relating to low performance and memory usage: #327 and #352.

[avently]

@themkat , thanks for the answer. The project I use here for testing is closed source, so can't share it. But I just googled a ktor example project we can both use for comparison of results. It's not using Android but produce the same errors.

git clone https://github.com/ktorio/ktor-samples
cd ktor-samples/chat

Then open this whole directory in nvim/VSCode/Emacs. Then choose specific file inside your app of choice: src/backendMain/kotlin/ChatApplication.kt.

In my case I see this:

First time I just opened this project without prebuilding it via command line. Then, after I saw errors, I built the project via
./gradlew assemble and reopened it again. Result was the same.

So have no idea what the problem is, but the project can be successfully built via the command I specified.

[themkat]

I can confirm that I also experience these errors... 😢 My guess it that it is because of this being a KMM project. I remember there was some other issue relating to KMM posted in the VSCode plugin: fwcd/vscode-kotlin#98 .
I tried printing all the package names that are resolved and going through them, and NO external dependencies found... (only reporting kotlin stdlib). My first guess would be that the Gradle classpath resolver need some fixing to work with KMM:
https://github.com/fwcd/kotlin-language-server/blob/main/shared/src/main/kotlin/org/javacs/kt/classpath/GradleClassPathResolver.kt
https://github.com/fwcd/kotlin-language-server/blob/main/shared/src/main/resources/kotlinDSLClassPathFinder.gradle

My guess is that it expects top-level dependencies-sections in the build.gradle files, and do nothing with the sourceSet declarations (found in the ktor one, and probably also your other one?). Have to familiarize myself with the gradle based part of the language server before I can think of a possible solution...

[avently]

My first guess would be that the Gradle classpath resolver need some fixing to work with KMM

Yeah, I see no usages of settings.gradle string in this repository which should be a main indication point of root gradle directory. If settings.gradle is found == it's a root directory of all modules. Maybe there are some other folder structures that allows multiple sub-sub-sub directories with multiple settings.gradle files but I didn't see such projects.

It's not only KMP problem. For example, opening this directory produce the same result: https://github.com/simplex-chat/simplex-chat/tree/stable/apps/android
It's just simple plain gradle project, single module.

[avently]

@themkat can you, please, share a link to any gradle project in the internet you're successfully using with this language server? Just want to try if I able to use it also.

[themkat]

It's not only KMP problem. For example, opening this directory produce the same result: https://github.com/simplex-chat/simplex-chat/tree/stable/apps/android
It's just simple plain gradle project, single module.

Hmm. that one I get errors in for one reason: not having installed the Android SDK (tons of warnings during the fetch dependencies stage). I guess your ANDROID_HOME env variable is set?

* What went wrong:
A problem occurred configuring project ':app'.
> SDK location not found. Define a valid SDK location with an ANDROID_HOME environment variable or by setting the sdk.dir path in your project's local properties file at '/Users/marie/Programming/Kotlin/simplex-chat/apps/android/local.properties'.

After installing it and fiddling around, I get these errors:

async0    Gradle task failed: [CXX1300] CMake '3.10.2' was not found in SDK, PATH, or by cmake.dir property.
[CXX1301] - CMake '3.23.2' found in PATH did not satisfy requested version.
[CXX1416] Could not find Ninja on PATH or in SDK CMake bin folders.
[CXX1300] CMake '3.10.2' was not found in SDK, PATH, or by cmake.dir property.
[CXX1301] - CMake '3.23.2' found in PATH did not satisfy requested version.
[CXX1416] Could not find Ninja on PATH or in SDK CMake bin folders.

Do you get any sort of similar errors? I had to fiddle a local.properties files in apps/android to set the properties. I could not find a specific cmake executable within the Android SDK, and I don't think fiddling with this for a long time is that useful. Seems a little wasteful to continue bloating up my system. Could not find that old of a cmake version with homebrew. I never do Android stuff, which is why I did not have any of the Android stuff installed 😆

(at least it gets much farther than the other KMM project! That one didn't even try to load any dependencies!)

I think the issue caused by this project is that it is a mix of both ndk and sdk usage, but I might be wrong.

@themkat can you, please, share a link to any gradle project in the internet you're successfully using with this language server? Just want to try if I able to use it also.

Sure. This project (the language server itself) works like a charm. Same with https://github.com/fwcd/kotlin-debug-adapter. You could also try to create a plain Kotlin app with https://start.spring.io/ .

[themkat]

Starting to wonder if either my setup is weird, or if the server is weird with Android apps. Tried this project and got this error during dependency resolution:

* What went wrong:
Unable to start the daemon process.
This problem might be caused by incorrect configuration of the daemon.
For example, an unrecognized jvm option is used.
Please refer to the user guide chapter on the daemon at https://docs.gradle.org/4.9/userguide/gradle_daemon.html

I'm probably not the best person to debug Android issues, but hopefully all of these comments either help future me or someone else who wants to try to debug it 😆

EDIT: Updated the Gradle wrapper version to 7.5, got no errors, but still NO resolved Android symbols... Weird. Only Kotlin stdlib reported in lsp output.

[avently]

@themkat

After installing it and fiddling around, I get these errors:

Just don't try to build this project, it's hard) For the CMake part the already built lib is needed which you don't have. I just opened it out of curiosity, saw errors and closed again:)

I found a test project for us, which is:

• using Android
• uses build.gradle instead of build.gradle.kts
• uses one gradle module
• doesn't use ancient gradle version.

https://github.com/keymapperorg/KeyMapper

[themkat]

Just don't try to build this project, it's hard) For the CMake part the already built lib is needed which you don't have. I just opened it out of curiosity, saw errors and closed again:)

I didn't try to build it manually, it is from the standard dependency resolution part of the language server (which runs gradle with a custom task I linked to above) 🙂 "Installing it" referred to Android SDK, as I don't code for Android (I dislike app development). If you can't run gradle to resolve dependencies, you can't do anything relating to external dependencies, it is that simple.

https://github.com/keymapperorg/KeyMapper

Get a cryptic error this time, due to class file major versions. Maybe it is because of me using Java 18. Have to investigate further later, as I have other things I need to do before getting back to work tomorrow.

[avently]

I guess your ANDROID_HOME env variable is set?

Yes, have set it for the entire environment:

env | grep ANDROID
ANDROID_HOME=/home/user/Android/Sdk

Get a cryptic error this time, due to class file major versions. Maybe it is because of me using Java 18

Probably, yes. In my case Java version is 11. Don't see errors related to SDK, but see unresolved dependencies. Kotlin keyword lazy unresolved too, so there is a problem with the language also.

Seeing errors like Cannot access built-in declaration 'Kotlin.Unit'. Ensure that you have a dependency on the Kotlin standart library.

P.S. file is app/src/main/java/io/github/sds100/keymapper/KeyMapperApp.kt

[themkat]

Probably, yes. In my case Java version is 11. Don't see errors related to SDK, but see unresolved dependencies. Kotlin keyword lazy unresolved too, so there is a problem with the language also.

Do you have a buffer like mine for Language server outputs? The errors I'm talking about is not shown as code errors, just in my lsp-log-buffer (or Kotlin-output section if using VSCode).

Seeing errors like Cannot access built-in declaration 'Kotlin.Unit'. Ensure that you have a dependency on the Kotlin standart library.

P.S. file is app/src/main/java/io/github/sds100/keymapper/KeyMapperApp.kt

I get none of those issues. lazy is found, no errors on Unit etc. Stdlib should be resolved automatically if it is present. On my machine it falls back to using Maven, and looks in my .m2-directory. From the language server output/logs:

async0    Successfully resolved kotlin-stdlib using Maven

stdlib should in general not be a problem, as long as it is either included in your dependencies (build.gradle or pom.xml) or is somewhere on your system (both Maven and Gradle should use .m2 if I'm not mistaken). There is a special stdlib resolver who tries a bunch of different things to fetch it.

Tried experimenting a bit now using Java 11. Had to install some packages in the Android SDK. This is from the language server output/logs (Gradle get dependencies step):

* What went wrong:
Execution failed for task ':app:kotlinLSPProjectDeps'.
> Failed to query the value of property 'bootClasspath'.
   > Failed to install the following Android SDK packages as some licences have not been accepted.
        build-tools;32.1.0-rc1 Android SDK Build-Tools 32.1-rc1
        platforms;android-32 Android SDK Platform 32

After installing those (had to do it manually for it to work), I had to wait for the symbol indexing populating to work, and the project just works 🙂 (this took almost 5 minutes on my M1 Macbook Air, which is insanely long for a project in my experience!). Only error is BuildConfig not being found. Completions work, and the external dependencies are loaded as expected. Without having my *lsp-log*-buffer handy (with Kotlin language server LOG statements), this would have been a nightmare to debug... 🙁

My conclusions is this:

• SDK location are fiddly here. I had to set it using sdk.dir in local.properties to get it working.
• Hidden missing Android SDK packages might be the cause of most issues in smaller applications like these. We should probably make those clearer, as I know there are several Android devs interested in using the language server, and feel pain with not seeing the server logs clearly. Unsure on what the solution here might be 🙁

Hope my stuff is helpful and not just spam 🙁 Not really that knowledgable on Android and its ecosystem... On the bright side, I'm learning something new 🙂

I will try to look more into the multi platform project issues (one of the first ones you sent, simplex-chat or ktor-samples chat) later this evening, or maybe tomorrow as I'm starting to get tired 🙂

EDIT: The ktor-samples chat issues, and other kotlin multi platform projects, seems to have another root cause. Still getting issues with external dependencies on those

[avently]

I cloned this repo (kotlin-language-server), opened nvim inside, got an error about missing maven, installed maven, reopened nvim, the project opened successfully. No errors, fast errors displaying after making some (right after living insert mode, even without delay). I was impressed.

Then I opened Keymapper project. I'm now much older than I was when did so. Project setup takes centuries. I stopped counting my age:) I mean, Indexing stage took more than an hour and still being indexing. Closed it, run ./gradlew assemble. Turns out I need to accept licenses as you are. Did it.

Now I got this error in lsp log file:

[WARN][2022-07-31 23:29:59] ...lsp/handlers.lua:456	"async0    Gradle task failed: \nFAILURE: Build failed with an exception.\n\n* Where:\nBuild file '/mnt/Dev/apps/keymapper/kotlin-language-server/server/src/test/resources/additionalWorkspace/build.gradle' line: 2\n\n* What went wrong:\nA problem occurred evaluating root project 'test-project'.\n> Could not get unknown property 'kotlinVersion' for object of type org.gradle.plugin.use.internal.PluginRequestCollector$PluginDependenciesSpecImpl.\n\n* Try:\n> Run with --stacktrace option to get the stack trace.\n> Run with --info or --debug option to get more log output.\n> Run with --scan to get full insights.\n\n* Get more help at https://help.gradle.org\n\nBUILD FAILED in 888ms"

First result in search is this: https://stackoverflow.com/questions/64834818/android-studio-could-not-get-unknown-property-kotlin-version

Results from this repo: https://github.com/fwcd/kotlin-language-server/search?q=kotlinVersion

Maybe after adapting language-server source code to suggestions from SO result will be better.

But at least it's something new. Which means installing maven helped somehow.

Do you have a buffer like mine for Language server outputs? The errors I'm talking about is not shown as code errors, just in my lsp-log-buffer (or Kotlin-output section if using VSCode).

There are so many lines in the lsp log (more than 160000), almost all of them about Back-end (JVM) Internal error when I try to open my projects.

Regarding sample chat, the log starts with this:

ERROR][2022-07-31 23:48:29] .../vim/lsp/rpc.lua:420	"rpc"	"/home/dev/.local/share/nvim/lsp_servers/kotlin_language_server/server/bin/kotlin-language-server"	"stderr"	'SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".\nSLF4J: Defaulting to no-operation (NOP) logger implementation\nSLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.\n'
[ERROR][2022-07-31 23:49:16] ...lsp/handlers.lua:454	"org.jetbrains.kotlin.codegen.CompilationException: Back-end (JVM) Internal error: Failed to generate function testSimpleConversation\nFile being compiled: (13,5) in //mnt/Dev/apps/ktor-samples/chat/src/backendTest/kotlin/ChatApplicationTest.kt\nThe root cause java.lang.IllegalStateException was thrown at: org.jetbrains.kotlin.codegen.state.KotlinTypeMapper$typeMappingConfiguration$1.processErrorType(KotlinTypeMapper.kt:128)"
[

After that multiple errors related to different things. In nvim it's still can't find Kotlin.Unit and others.

one of the first ones you sent, simplex-chat

As I said, better don't. To fully build the lib, you need to install Haskell, nix package manager, build GCC from source (which is a part of Nix build process), setup CMake and so on. It will take at least one day of your time. Would be better to get results with chat sample (since it's multiplatform and have a small codebase, while simplex-chat is not multiplatform). But it's your choice, of course. Maybe playing with Haskell and Nix will be interesting for you (as from the mathematics perspective) :)

[themkat]

As I said, better don't. To fully build the lib, you need to install Haskell, nix package manager, build GCC from source (which is a part of Nix build process), setup CMake and so on. It will take at least one day of your time. Would be better to get results with chat sample (since it's multiplatform and have a small codebase, while simplex-chat is not multiplatform). But it's your choice, of course.

Don't underestimate how stubborn I can be 😉 I ended up playing with the ktor chat example for now. I think I will soon have a working solution for resolving dependencies in kotlin multiplatfor projects, but it needs some more testing. The current solution I have implemented needs some tweaking to work, as it is far from perfect. Hope to get a PR done by this evening.

Maybe playing with Haskell and Nix will be interesting for you (as from the mathematics perspective) :)

Those are really fun to play with indeed 🙂 I've been wanting to go through the How To Solve It book my George Polya, and solve it in Haskell. Heard from someone long ago that it was really fun to do so. There is too much fun stuff to do for me to have time for everything 🙁