x/tools/gopls: performance regression on monorepos with gopls@v0.15.0+

[hitzhangjie]

What version of Go, VS Code & VS Code Go extension are you using?

Version Information

• Run go version to get version of Go from the VS Code integrated terminal.
  • go version go1.21.6 linux/amd64
• Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
  • Build info

---

golang.org/x/tools/gopls v0.15.2
golang.org/x/tools/gopls@v0.15.2 h1:4JKt4inO8JaFW3l/Fh9X1k/5JQn+iUOpdc4/Lpi0mOs=
github.com/BurntSushi/toml@v1.2.1 h1:9F2/+DoOYIOksmaJFPw1tGFy1eDnIJXg+UHjuD8lTak=
github.com/google/go-cmp@v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
golang.org/x/exp/typeparams@v0.0.0-20221212164502-fae10dda9338 h1:2O2DON6y3XMJiQRAS1UWU+54aec2uopH3x7MAiqGW6Y=
golang.org/x/mod@v0.15.0 h1:SernR4v+D55NyBH2QiEQrlBAnj1ECL6AGrA5+dPaMY8=
golang.org/x/sync@v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
golang.org/x/telemetry@v0.0.0-20240209200032-7b892fcb8a78 h1:vcVnuftN4J4UKLRcgetjzfU9FjjgXUUYUc3JhFplgV4=
golang.org/x/text@v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
golang.org/x/tools@v0.18.1-0.20240311201521-78fbdeb61842 h1:No0LMXYFkp3j4oEsPdtY8LUQz33gu79Rm9DE+izMeGQ=
golang.org/x/vuln@v1.0.1 h1:KUas02EjQK5LTuIx1OylBQdKKZ9jeugs+HiqO5HormU=
honnef.co/go/tools@v0.4.6 h1:oFEHCKeID7to/3autwsWfnuv69j3NsfcXbvJKuIcep8=
mvdan.cc/gofumpt@v0.6.0 h1:G3QvahNDmpD+Aek/bNOLrFR2XC6ZAdo62dZu65gmwGo=
mvdan.cc/xurls/v2@v2.5.0 h1:lyBNOm8Wo71UknhUs4QTFUNNMyxy2JEIaKKo0RWOh+8=
go: go1.21.6

• Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
  • 1.87.2
    863d2581ecda6849923a2118d93a088b0745d9d6
    x64
• Check your installed extensions to get the version of the VS Code Go extension
  • v0.41.2

• Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.

Share the Go related settings you have added/edited

Run Preferences: Open Settings (JSON) command to open your settings.json file.
Share all the settings with the go. or ["go"] or gopls prefixes.

"go.editorContextMenuCommands": {
"addImport": false,
"addTags": false,
"benchmarkAtCursor": true,
"debugTestAtCursor": false,
"fillStruct": true,
"playground": false,
"testAtCursor": false,
"testCoverage": false,
"testFile": true,
"testPackage": true,
"toggleTestFile": false
},
"go.formatTool": "default",
"go.goroot": "/usr/local/go",
"go.lintOnSave": "off",
"go.lintTool": "golangci-lint",
"go.testEnvFile": "",
"go.testEnvVars": {},
"go.testFlags": [
"-v",
"-count=1"
],
"go.toolsEnvVars": {},
"go.toolsManagement.autoUpdate": true,
"gopls": {
"formatting.local": "my repo module"
},

Describe the bug

My project is organized as monorepo including nearly 70 sub projects, its structure is as following:

myproject/
go.mod
go.sum
subproject1/ (no separate go.mod)
subproject2/ (no separate go.mod)
...
subprojectN/ (no separate go.mod)
vendor/

my development environment is:

• Host: windows11 + vscode (32GB RAM, CPU i7, 1TB SSD)
• Remote: vscode remote + Linux (cloud machine with 64GB RAM, 32Core CPU, 300GB local disk)
• The network is within the inner network of my enterprise, the latency is very low.

When I do some refactor, or jump to the definition, documentation, auto completion, it responds very very very slowly, nearly more than 30 seconds.

Steps to reproduce the behavior:

I cannot share the big monorepo to you, but I can investigate by myself if you could give me some suggestions about how to do.

Thank you.

[zhangfannie]

I encountered the same problem recently, the response was extremely slow.

[findleyr]

Hi, is this issue describing a recent regression, or has this always been a problem?

[findleyr]

To be more specific: do you experience this problem with gopls@v0.14.2?

go install golang.org/x/tools/gopls@v0.14.2

If so, then I think we can fix the regression. If not, there may still be things we can do to get gopls working better for you on your monorepo (albeit perhaps with some features being limited).

[zhangfannie]

I encountered the same problem recently, the response was extremely slow.

The issue I encountered was caused by a problem with GOROOT in my environment, and it is now working normally. Sorry to cause any disturbance.
BTW, I uses gopls@v0.15.2.
Thank you.

[findleyr]

Another thing you can do is share details about the gopls process when you perform one of these operations. Is gopls using a lot of CPU for the entire time you're waiting? How much memory does gopls use? This will help us confirm that the problem is indeed with gopls (most likely), and not something in the extension host.

[gordallott]

I don't want to hog someone else's (apparently resolved) issue but I have extreme monorepo performance issues with gopls, switching to gopls 0.14.2 resolved the issues

with any later version, completion is very slow (multiple seconds to bring up imports after using Add Import, multiple seconds to bring up symbols after typing <import>.. Saving a file takes about 5-6 seconds

there doesn't seem to by any memory/cpu use difference between 0.14.2 and other versions, the only difference seems to be that there are multiple go list commands that run and take a long time to return during pretty much any operation when using a version greater than 0.14.2

[adonovan]

@zhangfannie, do you also observe that the problem goes away when switching back to v0.14?

[findleyr]

It sounds like @gordallott's issues are related to goimports changes in gopls@v0.15.0+.

I think there's definitely something more going on here, so thank you @gordallott for chiming in. Can you share more about your workspace? Do you have multiple modules? Do you use go.work files?

Also, can you share an example of the go list commands you observe?

[gordallott]

sure, single module for the entire repo, some git submodules but not to other go repos. no go.work files

go list -tags debug,cloud -overlay=/tmp/gopackages-3706371482/overlay.json -e -json=Name,ImportPath,Error,Dir,GoFiles,IgnoredGoFiles,IgnoredOtherFiles,CFiles,CgoFiles,CXXFiles,MFiles,HFiles,FFiles,SFiles,SwigFiles,SwigCXXFiles,SysoFiles,TestGoFiles,XTestGoFiles,CompiledGoFiles,Export,DepOnly,Imports,ImportMap,TestImports,XTestImports,ForTest,DepsErrors,Module,EmbedFiles -compiled=true -test=true -export=false -deps=true -find=false -pgo=off -tags debug,cloud -- /home/gord/gordallott/axiom/... builtin

these go list cmds either don't execute with 1.14.2 - or they complete fast enough that my hacky ps monitoring didn't catch them

if i run that go list cmd with time, (and without the overlay because that part errors with file not found), i get

go list -tags debug,cloud -e  -compiled=true -test=true -export=false    -tag  3.00s user 1.72s system 280% cpu 1.682 total