Skip to content

UpdateProjectIfDirty is slow on large projects #60633

@dbaeumer

Description

@dbaeumer
Member

πŸ”Ž Search Terms

UpdateProjectIfDirty language server slow

πŸ•— Version & Regression Information

  • tested the version that ships with VS Code

⏯ Playground Link

No response

πŸ’» Code

Steps to reproduce:

  • clone VS Code (https://github.com/microsoft/vscode.git)
  • npm install
  • open VS Code on the vscode repository
  • open file task.constribution.ts
  • open the TS server log
  • add a new line to _ignoreEventForUpdateRunningTasksCount

Observe: updateGraphWorker takes roughly 300ms on my machine

I took a performance trace and a lot of time is spent in GC and getNormalizedAbsolutePath. I added a simply unbound cache to the method like

const pathCache = new Map();
function getNormalizedAbsolutePath(fileName, currentDirectory) {
  let cache = pathCache.get(currentDirectory);
  if (cache === undefined) {
    cache = new Map();
    pathCache.set(currentDirectory, cache);
  }
  let normalized = cache.get(fileName);
  if (normalized === undefined) {
    normalized = getPathFromPathComponents(getNormalizedPathComponents(fileName, currentDirectory));
    cache.set(fileName, normalized);
  }
  return normalized;
}

which cuts the time for updateGraphWorker in half. However there are still some GCs going on and a lot of time is now spent in verifyCompilerOptions / getNormalizedPathComponents.

πŸ™ Actual behavior

Takes 300ms to update on key stroke

πŸ™‚ Expected behavior

Adding a new line should be even in large project only cost a couple of ms :-)

Additional information about the issue

No response

Activity

dbaeumer

dbaeumer commented on Nov 28, 2024

@dbaeumer
MemberAuthor

Doing a comparable cache in getNormalizedPathComponents brings the time down to 75ms.

vscode-profile-2024-11-28-09-36-24.cpuprofile
vscode-profile-2024-11-28-12-42-42.cpuprofile
vscode-profile-2024-11-28-15-29-10.cpuprofile

Here are also some of the CPU profiles I captured.

added this to the TypeScript 5.8.0 milestone on Dec 2, 2024
DanielRosenwasser

DanielRosenwasser commented on Dec 2, 2024

@DanielRosenwasser
Member

This might make things a bit easier to iterate on.

git clone https://github.com/microsoft/vscode
cd vscode

npm ci --ignore-scripts
npm install @typescript/server-replay --no-save --ignore-scripts

cat > replay-script.txt << EOF
{"seq": 1, "type": "request", "command": "updateOpen", "arguments": { "changedFiles": [],"closedFiles":[],"openFiles":[{"file":"@PROJECT_ROOT@/src/vs/workbench/contrib/tasks/browser/task.contribution.ts","projectRootPath":"@PROJECT_ROOT@"}]} }
{"seq": 2, "type": "request", "command": "updateOpen", "arguments": { "changedFiles": [{ "fileName": "@PROJECT_ROOT@/src/vs/workbench/contrib/tasks/browser/task.contribution.ts", "textChanges": [{ "newText": "\n\t\t", "start": { "line": 146, "offset": 78 }, "end": { "line": 146, "offset": 78 } }] }], "closedFiles": [], "openFiles": [] } }
{"seq": 3, "type": "request", "command": "geterr", "arguments": { "delay": 0, "files": [ { "file": "@PROJECT_ROOT@/src/vs/workbench/contrib/tasks/browser/task.contribution.ts"} ] } }
EOF

npx tsreplay ./ replay-script.txt ./node_modules/typescript/lib/tsserver.js --logDir ./
DanielRosenwasser

DanielRosenwasser commented on Dec 3, 2024

@DanielRosenwasser
Member

@dbaeumer what platform have you experienced this on? On a Linux Codespace, I'm seeing something like 80ms. I'm guessing you might be hitting this on Windows though.

dbaeumer

dbaeumer commented on Dec 11, 2024

@dbaeumer
MemberAuthor

@DanielRosenwasser I was hitting this on WSL on my laptop. But even 80ms is a lot for a space in a file. When I looked at the code I was under the impression that most of the things that are done around path computations are 'unnecessary'.

DanielRosenwasser

DanielRosenwasser commented on Dec 12, 2024

@DanielRosenwasser
Member

Just to keep you updated, @andrewbranch will be looking into this.

andrewbranch

andrewbranch commented on Dec 12, 2024

@andrewbranch
Member

I get 95ms on WSL on a dev box. Should the difference between a dev box and your laptop be 3x+? @dbaeumer can you send a TS Server log?

dbaeumer

dbaeumer commented on Dec 16, 2024

@dbaeumer
MemberAuthor

I can produce a server log. But even 95 ms is IMO a lot considering that most of the work is around normalizing paths that are already normalized.

dbaeumer

dbaeumer commented on Dec 16, 2024

@dbaeumer
MemberAuthor

Here is a tsserver.log from my laptop running on a power adapter. I see times between 170 and 300 ms.

tsserver.zip

DanielRosenwasser

DanielRosenwasser commented on Dec 16, 2024

@DanielRosenwasser
Member

Thanks, we'll take a look!

Just to test things out, #60754 (comment) and #60755 (comment) each have a build you can try out. Can you give us a sense of if either of these fix the issue on your side? If you can profile while running to give us a sense of where the remaining major work is, that would be helpful.

dbaeumer

dbaeumer commented on Dec 17, 2024

@dbaeumer
MemberAuthor

Times for build 60754 were between 78 - 150 ms with server plugins enabled.
Times for build 60755 were between 66 - 109 ms with server plugins enabled.

12 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Needs InvestigationThis issue needs a team member to investigate its status.

Type

No type

Projects

No projects

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @DanielRosenwasser@dbaeumer@andrewbranch@jakebailey@RyanCavanaugh

      Issue actions

        `UpdateProjectIfDirty` is slow on large projects Β· Issue #60633 Β· microsoft/TypeScript