Overview of approaches on a language server #11
Comments
|
So none of them are using Svelte parser? |
|
svelte2tsx strips out style/script tags and then uses the svelte compiler to parse the htmlx part. |
|
I wasn't sure if I should share this, but some part of it might be useful. Last year (July/August 2019) I started playing around with typechecking Svelte, knowing that I was doing throwaway work just to learn. It supports some unknown amount of a full solution, maybe 30%, and it has some limitations (more on those later). There aren't any tests or external documentation because I've always considered it to be a dead end. Svelte compilation goes as it does today with a non-typechecking TS preprocessor. If you wanted types in your HTMLx markup, not just your script tag, you'd need a preprocessor to handle that, or Svelte would need to handle TS internally. The typechecker is a separate build step that uses the un-preprocessed TypeScript code from The TypeScript compiler API is used to construct a program in memory and typecheck the virtual Svelte TS. Unlike Svelte its view of the world is whole programs, not individual files. At the time it seemed like it would be straightforward to add incremental compilation and worker threads as future optimizations. The virtual Svelte TS file is generated with a sourcemap which is then used to map TypeScript diagnostics back to the original Svelte source. It was pretty cool to see type errors point back to the Svelte source but I had no confidence in my approach overall. One problem I remember that I couldn't figure out was with diagnostics that weren't specific enough in some cases. IIRC an example was with typechecking Svelte component constructor props - VSCode somehow would point to individual prop errors, but the TS diagnostics I was getting were pointing only to the constructor, not the problem property. If you look through the code be aware that parts are a mess and some comments may be outdated. I originally used Here are the 4 files referenced above: |
|
I did some digging in the source code of the current svelte language server. Here are my thoughts on the typescript part of it: OverviewThe language server broadly works likes this:
Current shortcomings and ideas for improvement
Thoughts? |
|
On |
|
@dummdidumm do you mind acting as tech lead and breaking these things out to specific issues? i would be happy to volunteer dev time, but i find alot of this overwhelming and would work better in small, defined issues. Unless there is some big architectural change that needs to be done (looks like no?) Basically just open up a bunch of issues and let people claim them and manage them as our informal tech lead on this. I'd offer to do it but honestly just havent felt the pain yet haha also linking to orta's original LSP post for others who are browsing sveltejs/svelte#4518 |
|
Sure I would love to help on this, but right now I'm just a random dude who is very excited about the language server, digging into the code and making some suggestions 😄 I also need some more time to get familiar with the code, its architecture and lsp in general. But I'm definitely willing to push stuff forward. @orta since you are the maintainer of this repo, what do you think about this? |
|
👋 Yeah, we both are random people who just care - I don't even have any svelte codebases to trigger my "this should be better" itch. So, I'm very happy to see any movement at all - @dummdidumm - push away and I'll back you up and try make sure everything runs smoothly |
|
Ok great! I will go ahead and make some PRs restructuring some of the code and preparing it so we can then parallelize working on individual items. |
|
Maybe we could combine @ryanatkn, svelete-ts's approach and eslint-plugin-svelte3's variable approach:
The script can have three fragments: module Module vars, where In this way we don't have to re-implement on how to find reactive variable About reactive variable $: a = 1let a: number
$: a = 1or $: a = someFunction(b) as TypeAll it's valid ts that doesn't need transform before transpile And $-prefixed store we can create a generic function like svelte/store get to extract store type /** injected */
let $store = getType(store) |
I think svelte-ts does something like that. Seems like a good idea. On
So you want to split up the file into a virtual
Is it possible to infer the type? Maybe this is just something we leave as is and if the user wants to use typescript, he has to define
Yeah this seems nice. Another idea I had was to construct getters/setters. |
The compile result has a property vars, which is an array of object that has the following property:
This is the approach of eslint-plugin-svelte3. Also I just thought if we want to support ts in template because there is no attribute to specify language.
Since this virtual source is only use for type check and autocomplete, I presume it can achieve by just copy the first assign statement to initialize it but it seems to be need more work. |
Looks like normal inference doesn't get us all the way there. (did you mean compile-time inference? not sure if that could work) let a; // type is "any" initially
$: a = 5;
a; // type is now "number", good
const cb = () => a; // type is implicitly "any", nooThe reason is that in the linear code flow, TypeScript can see |
|
Exactly, these were my worries with that. But as @jasonlyu123 pointed out, we could just copy the definition (everything after the Before $: a = true;After let a = true;
$: a = true;But I don't know if that's feasible for all circumstances. Especially with large objects where someone implements an interface, this will not infer everything as the user might expect. Also, the copy-pasta could get pretty big. So I'm still more on the side of "let the user type it". |
|
Thanks for clarifying, I misunderstood what @jasonlyu123 said. It fixes the nested function scope issue I mentioned. Is there an example of when that's not good default behavior? It seems like it does the ergonomic right thing most of the time, and other cases can be fixed manually. Here's some examples. |
|
Mhm I guess you're right. It is the most ergonomic thing. What I worried about is if the inferred type is wrong, for example property is |
|
There's a part of me that's like: "great, the explicit let syntax works awesome" but that would end up not working with JS support out of the box |
|
What about just replace first label to let: $: a = 1
$: a = 2get transform to /** injected */
let a = 1
$: a = 2I rethink a bit and can't find the advantage of "copy definition" over "replacing About inferred type difference to what user expected, can we provide a refactoring actions to convert it to manual define? This action will prompt user to manual type the variable like this, Or refactor to its inferred type if possible. before $: a = 'const1'after let a : AskUserForType
$: a = 'const1'Also thinks for the things you point out, Learned something 👍 |
|
Ha, smart idea, just entirely replace the |
|
Nice, I can't think of any reason replacing the label would fail either. (since For redeclaring the same reactive variable, it looks like Svelte allows this and it works in order like you'd expect. (see this repl example) I don't personally see myself using this pattern intentionally and would want the error, but it's in the territory of interpreting Svelte's semantics so don't take my word for it! |
|
OK so redeclaring it is a pattern that actually works. I would say we better allow it then - or make a "redeclare not allowed"-feature flag later which is off by default. |
|
@halfnelson has created a language server based off his svelte2tsx approach in https://github.com/halfnelson/svelte-type-checker-vscode . I really love how good this all works already, but I still would prefer to not use tsx as an intermediate output because I fear it could be a little too slow and have some limitations - but that's just my gut feeling. Still, we could take a lot from svelte2tsx: How the flow of parsing-compiling-transformation is done, and also the extensive test suite. How do others see the approach of transforming the code to tsx? |
|
I though about how other transformations could look like. Here are some examples. Feedback welcome. How a component definition looks likeOriginal: <script>
import { createEventDispatcher } from 'svelte';
const dispatch = createEventDispatcher();
export let todo: string;
function doneChange() {
dispatch('doneChange', true);
}
</script>Converted: export default class Component {
constructor(props: {todo: string; 'on:doneChange': (evt: CustomEvent<boolean>) => void}) {}
}
import { createEventDispatcher } from 'svelte';
const dispatch = createEventDispatcher();
export let todo: string;
function doneChange() {
dispatch('doneChange', !todo.done);
}Explanation: (in the following examples, component-definition is omitted for brevity) Use variable in templateOriginal: <script>const bla = {bla: 'bla'};</script>
<p>{bla.bla}</p>Converted: const bla = 'bla';
const _bla = bla.bla;Explanation: Use native event listenerOriginal: <script>function bla() {return true;}</script>
<button on:click={bla}>bla</button>Converted: function bla() {return true;}
const btn = document.createElement('button');
btn.addEventListener('click', bla);Explanation: Use other componentOriginal: <script>
import Bla from './bla.svelte';
function blub() {}
</script>
<Bla bla={1} on:blubb={blub} />Converted: import Bla from './bla.svelte';
function blub() {}
const bla = new Bla({bla: 1, 'on:blubb': blub});Explanation: Each loopOriginal: {#each todos as todo,i (todo.id)}
<p>{todo.text}</p>
{/each}Converted: todos.forEach((todo, i) => {
const _todoText = todo.text;
});Explanation: IfOriginal: {#if x === 1}
<p>if</p>
{else if x === 2}
<p>elseif</p>
{else}
<p>else</p>
{/if}Converted: if (x === 1) {
} else if (x === 2) {
} else {
}Explanation: Missing pieces
What do you guys think? Are these transformations viable and doable? Did I miss something? |
|
About component class I would prefer extends svelte's own SvelteComponent class, but i don't know if this is easy to implement or not import { SvelteComponent } from "svelte";
export default class Component extends SvelteComponent {
constructor(options: ComponentOption<{ propA: string }>) {
super(options)
}
$on(
event: 'input',
callback: (event: CustomEvent<string>) => void
): () => void
$on(
event: 'click',
callback: (event: CustomEvent<number>) => void
): () => void
$on(
event: string,
callback: (event: CustomEvent<any>) => void
): () => void {
return () => { }
}
}and import these interface interface ComponentOption<TProps, TSlot = undefined> {
target: Element,
props: TProps | SlotProp<TSlot>
}
type SlotOption<T> = [unknown, unknown, unknown]
interface SlotProp<TSlot> {
$$slots: Record<string, SlotOption<TSlot>>
}Noted The reason for this is that it can then be use to emit <script>
onMount(() => {
new Component({
target: document.getElementById('foo')
})
})
</script>Element directives
fade(null as Element, { } /* option in attribute*/)Await{#await promise}
<p>...waiting</p>
{:then number}
<p>The number is {number}</p>
{:catch error}
<p style="color: red">{error.message}</p>
{/await}to promise.then(number => { number })
promise.catch(error => { error.message })bind:this<script>
let a;
</script>
<div bind:this={a}><div>to let a;
onMount(() => {
a = document.createElement('div')
})needed to wrap it in callback because it's not immediately available context="module"This is a bit tricky <script context="module">
let count = 1;
</script>
<script>
import _ from "lodash"
let b;
</script>should be compile to import _ from "lodash"
let count = 1;
// any block is ok
{
let b;
{And also what if the user somehow use different language in two script, like js in module and ts in instance? |
|
Nice, looking good. For "Use variable in template" |
|
Some of the Vue team are exploring a typescript language server plugin - https://github.com/znck/vue-developer-experience It's an interesting angle as today these experiences go from outside TypeScript and in (like vetur does, as it runs it's own tsserver-ish, as well as html/css/etc LSPs) or do you try go from inside TypeScript and work outwards (e.g. this proposal) where you manipulate the TSServer to effectively believe that the .vue files are legitimate TypeScript by masking non-TS code. Though today it can only work by having patches to typescript |
|
That's an interesting approach. But how would features like autocompletions for html or css work then? Would one have to implement this all "by hand" and not rely on the out-of-the-box-language-servers anymore, or am I missing something? Do you see any advantages of doing one over the over? |
|
Anything but the JS/TS support would be handled by a separate vscode LSP. It would be like if this extension removes all support for JS/TS and only handled the rest, then the typescript plugin would handle the rest. Advantage wise you get all of the TS tooling "for free" in this case, jump to symbol, file mappings etc I'm probably not recommend it, today it requires forks or patches of TypeScript to work which is a pretty high barrier to entry - and the TypeScript team is still not sure if/when compiler plugins could have that kind of access. |
|
Thanks for the insights. I would say we with the current approach then (outside-in). If we go with the "make a virtual ts file"-approach discussed above, how would we implement this and keep source mappings? Today, it's easy because it's just "find start of script, find end of script", "extract script part" and for mapping "add offset of svelte file to positions". Original <script lang="typescript">
let a = 1;
$: b = a + 1;
</script>
<p>{b}</p>Mapped: export default class Bla extends SvelteComponent { ... } // <- prepended, no mapping needed as far as I know
let a = 1; // <- untouched
let b = a + 1; // <- modified in place. How to map? Do we need to adjust all following code positions now to have a new offset/position?
b; // <- appended. Needs mapping from {b} inside svelte-mustache-tag to here. We can get the original position from the html ast which is part of the output of svelte.parseAny ideas? I never did source mapping stuff before.. |
The remove TypeScript patch PR is merged. |
|
So ... @orta this means the disadvantages regarding "unofficial" are no longer the case? Or is it still kind of unofficial, just not through a patch but through some Another topic: At the moment I feel we still are in discussion on how to approach this best, and if we decide, then it's going to take some time to implement it. What do you guys think of using |
|
Aye, there were two ways it was editing TS's default flow, It's still patching TypeScript - but at runtime by replacing TS' functions. I think it's reasonable, given plugin development isn't on the roadmap yet. I'd be a bad core team member if I recommended it though ;) Trying out |
|
You can see what it would be like with svelte2tsx using this vscode plugin: https://marketplace.visualstudio.com/items?itemName=halfnelson.svelte-type-checker-vscode together with Svelte Beta (works best if you disable svelte beta's typescript options so you don't get double the hints :) ) |
|
Even if not using svelte2tsx, I see discussion on which transforms need to be made to svelte's JS code to keep typescript happy. The test suite for svelte2tsx covers not only changes needed to be made to the template, but also the script tag, which if you are going for a script tag only solution might be a good starting point: https://github.com/halfnelson/svelte2tsx/tree/master/test/svelte2tsx/samples the |
|
Just discovered this repo https://github.com/pivaszbs/svelte-autoimport . Maybe worth looking into if we want import intellisense independent of |
|
I think we're at a point where most of the general direction for this project is sorted and seems to be working out neatly. I'll give this a week for feedback, but I'm pretty happy that we don't need to re-think it architecturally now. |
|
Seems noone has objections, so I'm gonna close this one. Thanks everyone for discussing this! |
This thread is intended for an overview of approaches on how to implement a language server for svelte files. Discussion on a prefered solution can be done here aswell.
Current state of the language server
Currently syntax highlighting and some basic autocompletion works for svelte files.
Rich intellisense however is not provided, does not know about all the typescript files in the workspace, and also has no type information about the svelte components.
The same is true for the html part where special svelte syntax is highlighted but intellisense is not working. The vscode-html-languageservice is used which does not know about the special syntax by default.
The current language server does use the svelte compiler for parsing files and getting diagnostics like "no alt attribute on this img tag".
A more indepth analysis: Comment
Existing approaches/solutions
Disclaimer: I'm not the author of any of the existing solutions, I did look at the code however. If some of the information is wrong or not detailed enough, or you are missing a solution, feel free to comment, I will adjust it.
https://github.com/alexprey/sveltedoc-parser
Uses htmlparser2 to parse the html part of a svelte file. Uses hooks of the html parser to add custom parsing and extract relevant information.
Uses espree to parse the script parts of a svelte file. Then walks the AST to extract relevant information.
Provides the results in a JSON-Format.
https://github.com/ArdenIvanov/svelte-intellisense uses it to parse svelte files. It uses the result to attach some additional behavior.
Since this uses a javascript parser, it will not work with typescript.
https://github.com/halfnelson/svelte2tsx
Uses svelte's compiler to parse the HTMLx part of a svelte component. Then transforms the HTMLx plus the original script part to a tsx file with a self-written transformer.
https://github.com/simlrh/svelte-language-server/blob/feature/extend-ts-support/src/plugins/ts-svelte/service.ts (a fork from svelte-language-server) uses it to create shadow-tsx-files of svelte-files. It then uses typescript's language service, but proxies the requests: It replaces the original file path with the file path to the generated tsx-file. The typescript language server therefore checks the generated tsx files. Results are transformed back to get correct document positions.
https://github.com/halfnelson/svelte-type-checker-vscode is another language server using the svelte2tsx-library.
https://github.com/marcus-sa/svelte-ts
Description taken from implementer's comment:
Compiler:
It consumes the exported variables and functions from inside a Svelte script, and then generates it as a declaration files which contains a class that extends the runtime SvelteComponent.
Typechecker:
Other approaches
See comments of this thread.
The text was updated successfully, but these errors were encountered: