profile
viewpoint

heschik/govim 0

govim is a Go development plugin for Vim8, written in Go

heschik/tools 0

[mirror] Go Tools

issue commentmicrosoft/vscode-go

Autocompletion for symbols from non imported packages without typing package name

Sure, doing it for packages that have been loaded would be relatively easy. I don't think we'd want to do it for random packages in the module cache.

ghost

comment created time in 2 days

issue commentmicrosoft/vscode-go

Go plugin makes cgo spam the system

This isn't totally shocking behavior, but I wasn't able to reproduce it. Could you gather a gopls log and attach it? I'm particularly interested in packages.Load calls.

abergmeier

comment created time in 2 days

issue closedgolang/go

x/tools/gopls: adds import for package not required by module dependencies

Please answer these questions before submitting your issue. Thanks!

What did you do?

See https://play.golang.org/p/tNlNNlBk6lx for the full listing.

Edit a module that requires github.com/aws/aws-sdk-go module, refer to some package symbols from that module with an empty "import" section:

sess, err := session.NewSession()
if err != nil {
	return err
}
svc := cloudwatchlogs.New(sess)

Save file in VSCode to trigger automatic import fixes with gopls.

What did you expect to see?

Import section populated with packages from dependencies:

"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/cloudwatchlogs"

What did you see instead?

Import section incorrectly populated with package that's not listed in dependencies (second line):

"github.com/aws/aws-sdk-go/aws/session"
"github.com/awslabs/fargatecli/cloudwatchlogs"

Build info

golang.org/x/tools/gopls v0.3.2
    golang.org/x/tools/gopls@v0.3.2 h1:eP1aj1AvT6ynElQH6KP0mmOT2gnWa1gYclHL4wGUbMo=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
    golang.org/x/tools@v0.0.0-20200212213342-7a21e308cf6c h1:D2X+P0Z6ychko7xn2jvd38yxQfdU0eksO4AHfd8AWFI=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info

go version go1.14rc1 darwin/amd64

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/artyom/Library/Caches/go-build"
GOENV="/Users/artyom/Library/Application Support/go/env"
GOEXE=""
GOFLAGS="-ldflags=-w -trimpath"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/artyom/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/artyom/Library/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/artyom/Library/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/artyom/Downloads/gopls-bug/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/lb/3rk8rqs53czgb4v35w_342xc0000gn/T/go-build534508186=/tmp/go-build -gno-record-gcc-switches -fno-common"

closed time in 3 days

artyom

issue commentgolang/go

x/tools/gopls: adds import for package not required by module dependencies

Duplicate of #36077, I'm afraid. If you use autocomplete, rather than organize-on-save, it will find the right package. We might be in a position to make some progress now, I'll think about it when I get a chance.

artyom

comment created time in 3 days

issue commentgolang/go

x/tools/internal/fastwalk: "checkptr: unsafe pointer conversion" during darwin race test

@mdempsky @randall77 I could use some help with this one.

There are two possible cases here: https://github.com/golang/go/blob/56d6b87972c9852570fe017ac5fa153314c21992/src/runtime/checkptr.go#L9-L21

(It would be lovely if they had different error messages. Should I send a CL?)

I think the first case is trivially ruled out by the stack trace -- 0xc0002c1c08 is maximally aligned, right? So my best guess is that the Darwin Getdirentries implementation has returned a bad slice. Am I missing something?

leitzler

comment created time in 3 days

issue commentgolang/go

proxy.golang.org: returns not found for @v/list of a module deleted from the origin

The above looks about right to me. In general we will tolerate failures for a little while, so where you say "fails" it really means "fails for long enough that we give up." 8 is a little confusing: @latest is specified to only return one thing.

If you have more questions let's find somewhere else to go through them than this semi-arbitrary bug :)

cuonglm

comment created time in 6 days

issue commentgolang/go

x/tools/gopls: using a lot of memory

Can you check your /tmp directory for files named gopls.*? It should have written out some profiles. They're named after the size of the heap; please upload the ones from the biggest heaps.

ajeecai

comment created time in 7 days

issue commentgolang/go

cmd/go: "package not found" error is less useful than GOPATH's when the missing package could have been in the main module

Also, I'd like to see the same thing for replace targets, not just the main module.

heschik

comment created time in 7 days

issue commentgolang/go

cmd/go: "package not found" error is less useful than GOPATH's when the missing package could have been in the main module

Sure, I didn't mean to dictate any specifics. I was more arguing for a philosophy of error reporting: for each location looked up, there should be some variety of not found error, and if resolution eventually fails, the resulting message should detail each location searched and why it didn't work. So for the case at the end of your comment, I would hope for something like ../gopath/src/example.com/mypkg (module example.com): is hidden by nested module example.com/mypkg. I'm assuming things about the structure of the code, of course.

heschik

comment created time in 7 days

issue openedgolang/go

cmd/go: module mode's "package not found" error is less useful than GOPATH's

Given the following setup:

-- gopath/src/example.com/go.mod --
module example.com

go 1.14
-- gopath/src/example.com/main.go --
package main

import (
        "fmt"

        "example.com/mypkg"
)

func main() {
        fmt.Println(mypkg.Message)
}
-- gopath/src/example.com/mypackage/doc.go --
package mypackage

const Message = "sup"

and then running:

$ export GOPATH=$PWD/gopath
$ cd gopath/src/example.com
$ GO111MODULE=off go run main.go
main.go:6:2: cannot find package "example.com/mypkg" in any of:
	.../go/src/example.com/mypkg (from $GOROOT)
	../gopath/src/example.com/mypkg (from $GOPATH)

This is clear and actionable: there are two directories that could have contained the package, and neither did. You can ls the directories, compare the paths to the filesystem, etc.

In module mode:

$ GO111MODULE=on gotip run main.go
go: finding module for package example.com/mypkg
main.go:6:2: cannot find module providing package example.com/mypkg: unrecognized import path "example.com/mypkg": reading https://example.com/mypkg?go-get=1: 404 Not Found

This is much less actionable. Why is it going to the Internet for a package that should be on the local filesystem? Did it look anywhere first, and if so where? (If example.com were a GitHub URL the error would be different, but talking about private repos isn't any more helpful here.)

In case it wasn't obvious, the problem is a typo: the package is named mypackage, not mypkg. I think the GOPATH error gives a new user a fighting chance of finding their mistake; the module error needs at least a basic understanding of package-to-module resolution, and nested modules (!) to make any sense.

Concretely, I would like to see the module mode error look more like the GOPATH error:

main.go:6:2: cannot find package "example.com/mypkg" in any of:
   	.../go/src/example.com/mypkg (standard library)
	../gopath/src/example.com/mypkg (module example.com)
       https://example.com/mypkg (module example.com/mypkg): unrecognized import path "example.com/mypkg": reading https://example.com/mypkg?go-get=1: 404 Not Found

@bcmills @jayconrod

created time in 8 days

issue closedgolang/go

proxy.golang.org: returns not found for @v/list of a module deleted from the origin

What version of Go are you using (go version)?

<pre> $ go version

</pre>

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env

</pre></details>

What did you do?

$ curl https://proxy.golang.org/github.com/belogik/goes/@v/list
not found: github.com/belogik/goes@latest: git ls-remote -q origin in /tmp/gopath/pkg/mod/cache/vcs/0c31a160b38443d5e33ca2a5e60e51734d83293b66dc3d9b0c12699f8d2b5cec: exit status 128:
	fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

What did you expect to see?

List of available versions.

What did you see instead?

Error.

The reason seems due to github.com/belogik/goes was recently removed by the author. If I query for specific version, proxy still returns the result.

$ curl https://proxy.golang.org/github.com/belogik/goes/@v/v0.0.0-20151229125003-e54d722c3aff.info
{"Version":"v0.0.0-20151229125003-e54d722c3aff","Time":"2015-12-29T12:50:03Z"}

closed time in 8 days

cuonglm

issue commentmicrosoft/vscode-go

automatic import adder should not separate first import from its comment

Seems someone got to it in the intervening 2 years.

$ cat foo.go
package main

//#include <stdlib.h>
import "C"

func main() {
    fmt.Print()
}
$ goimports foo.go
package main

//#include <stdlib.h>
import "C"
import "fmt"

func main() {
	fmt.Print()
}
mbenkmann

comment created time in 9 days

issue commentgolang/go

x/tools/gopls: high memory usage in sigs.k8s.io/cluster-api

@jvburnes Loading stuff takes time, so I'm not sure what you mean by it starting at 40MiB. When I open https://github.com/dexidp/dex, I see gopls stabilize at an RSS of about 600MiB. If you're seeing dramatically more, please provide repro instructions and/or profiles.

vincepri

comment created time in 9 days

issue commentgolang/go

x/tools/gopls: Load concurrency error reported to LSP client

I would be happy to make it a warning but we don't have those yet.

myitcv

comment created time in 10 days

issue commentgolang/go

x/tools/gopls: incorrect diagnostics when navigating through 'git' and 'gitlens' files

There's a lot of mess under the covers here.

  • span.NewURI doesn't account for non-file schemes. If you pass it a gitfs URI, it puts file:// in front of it.
  • span.(*URI).Filename, with 102 call sites, panics if you call it on a non-file URI.

Putting those two facts together, you can't reliably tell what kind of URI you've got, and if you fix NewURI to not mangle its results, then you get panics all over the place when one pops up somewhere unexpected.

I think the only feasible option is to reject non-file URIs at the edges. Will think more.

inliquid

comment created time in 10 days

issue commentgolang/go

x/tools/gopls: incorrect diagnostics when navigating through 'git' and 'gitlens' files

@dskdas You can restart the LSP with ctrl-shift-p, then Go: Restart Language Server.

Notes: An easy to way to reproduce this is to rename some commonly-used type, then open a file affected by the rename in the Source Control pane. VS Code sends a didOpen for a gitfs: URI, which we treat as if it were a regular file URI. From there it's a bit of a comedy of errors:

  • We do a file= query for some insane combination of the workspace root and the full gitfs URI (?!). Not surprisingly, it finds nothing.
  • We don't get a zillion duplicate definition errors, probably because we don't include it when type-checking the package.
  • Somehow the file is type checked enough to produce undeclared name errors. I'm confused by this bit.

On the one hand, a lot of features work surprisingly well in the gitfs: file -- basically anything that references a symbol outside the file itself. On the other, this is conceptually impossible: literally anything could have happened to the project since whatever version of the file you're looking at, so there's no reason to expect anything to still exist.

Just suppressing diagnostics in these files might be good enough to paper over a lot of the problems. Let me see how bad the experience is after that.

inliquid

comment created time in 10 days

issue commentgolang/go

x/tools/gopls: high memory usage in sigs.k8s.io/cluster-api

All: I just merged https://golang.org/cl/218858, which will automatically write profiles when gopls uses more than 5GiB of memory. If you'd like, you can update to master with go get golang.org/x/tools/gopls@master golang.org/x/tools@master and then attach the heap and goroutine profiles for us to look at.

vincepri

comment created time in 10 days

issue commentgolang/go

x/tools/gopls: create link for release note for each version

Done. I vote we keep the notes from the most recent minor series at https://github.com/golang/go/issues/33030#issuecomment-510151934, as I've done now.

hyangah

comment created time in 10 days

issue commentgolang/go

x/tools/gopls: releases

v0.3.1

  • https://github.com/golang/go/issues/36975: Handle nil pointer in builtin packages if workspace load fails.
  • https://github.com/golang/go/issues/36999: Fix panic on files that contain % in them.
  • Fix memory leak caused by autocompletion (CL 217677). Users were seeing huge memory usage over long sessions. Please comment on https://github.com/golang/go/issues/36943 if you're still seeing this issue with gopls/v0.3.1.
stamblerre

comment created time in 10 days

issue commentgolang/go

x/tools/gopls: create link for release note for each version

Perhaps we should just edit the latest minor version's release notes into the top comment.

hyangah

comment created time in 10 days

pull request commentgolang/tools

update documentation semver package

Feel free to add me once the bot has created a Gerrit CL -- we don't review PRs directly.

I think the quoting on your reply might have confused the bot, since it still has you as having not signed the CLA.

cohental

comment created time in 11 days

issue commentgolang/go

runtime: 10ms-26ms latency from GC in go1.14rc1, possibly due to 'GC (idle)' work

I think this is working as expected. At the heart of github.com/golang/groupcache/lru is a single large map. AFAIK the map has to be marked all in one go, which will take a long time. In the execution trace above you can see a MARK ASSIST segment below G21. If you click on that I think you'll find that it spent all of its time marking a single object, presumably the map. Austin and others will know better, of course.

Since a real server would need to lock the map, sharding it would be a good idea, which would also reduce the latency impact of a mark.

(cc @rsc, since I think @cagedmantis made a typo.)

thepudds

comment created time in 12 days

issue commentgolang/go

x/tools/gopls: imports.Resolver is *imports.gopathResolver, not *imports.ModuleResolver

Thanks, that makes sense. I suspect the quoting is a red herring -- your script should probably have had "$*" at the end, not just $* -- but we should not be passing build flags to go env at all. I made that change in https://github.com/golang/tools/blob/61798d64f0258df3b7b6e3667997eaebec9c0d99/go/packages/golist.go#L710 but we have too many copies of invokeGo floating around.

Glad it's working for you for the moment, I'll clean it all up when I get a chance.

gwillem

comment created time in 12 days

issue commentgolang/go

x/tools/gopls: Cannot install 0.3.1 with go get

You're on Fedora, I presume, which disables proxy.golang.org by default, making the go command vulnerable to #34092. You can temporarily avoid the bug by setting GOPROXY=proxy.golang.org,direct, running with GOPATH=$(mktemp -d), or clearing your cache with go clean -modcache. The issue should be fixed in 1.14.

paulbdavis

comment created time in 13 days

issue commentgolang/go

x/tools/gopls: imports.Resolver is *imports.gopathResolver, not *imports.ModuleResolver

Still very strange.

There should be more than one call to go env. The one that matters is this one: https://github.com/golang/tools/blob/61798d64f0258df3b7b6e3667997eaebec9c0d99/internal/imports/fix.go#L793 which only reads GOMOD and doesn't pass -json. Something strange is happening with it that's not happening with the others.

Now that I'm looking at it, swallowing the error is a bad idea. I'll look at fixing that next week. If you'd like, figuring out what's going wrong with it will get this resolved more quickly.

gwillem

comment created time in 13 days

issue commentgolang/go

proxy.golang.org: returns not found for @v/list of a module deleted from the origin

The fix ended up being somewhat more complicated than I thought. We'll probably have it rolled out toward the end of next week.

cuonglm

comment created time in 13 days

issue commentgolang/go

x/tools/gopls: imports.Resolver is *imports.gopathResolver, not *imports.ModuleResolver

Very strange. I could fix the crash very easily, but you'd be left with broken autocomplete. The problem is that the goimports code used for autocomplete thinks you're in GOPATH mode. I'm not sure why.

A couple questions: Can you show us your settings? I'm particularly interested in any changes to the environment, working dir, etc. If you set "gopls": {"verboseOutput":true} in your settings, the first time you type in a Go file there will be a couple log lines like:

[Info  - 5:29:33 PM] 2020/02/07 17:29:33 11.878056ms for GOROOT=... GOPATH=... GO111MODULE= GOPROXY=https://proxy.golang.org,direct PWD=... go [go env GOMOD]

Can you show us those too? Which directory do you start VS Code in?

gwillem

comment created time in 13 days

issue commentgolang/go

proxy.golang.org: returns not found for @v/list of a module deleted from the origin

After internal discussion, we think this behavior of proxy.golang.org is, if not an outright bug, undesirable. The module author is free to set GOPRIVATE and do whatever they need to do, but downstream users can't add a dependency on the module without knowing a valid pseudoversion somehow.

$ go get -x github.com/belogik/goes@latest
# get https://proxy.golang.org/github.com/@v/list
# get https://proxy.golang.org/github.com/belogik/@v/list
# get https://proxy.golang.org/github.com/belogik/goes/@v/list
# get https://proxy.golang.org/github.com/belogik/goes/@v/list: 410 Gone (0.165s)
# get https://proxy.golang.org/github.com/@v/list: 410 Gone (0.269s)
# get https://proxy.golang.org/github.com/belogik/@v/list: 410 Gone (0.270s)
# get https://github.com/?go-get=1
mkdir -p /.../pkg/mod/cache/vcs # git3 https://github.com/belogik/goes
# lock /.../pkg/mod/cache/vcs/0c31a160b38443d5e33ca2a5e60e51734d83293b66dc3d9b0c12699f8d2b5cec.lock# /.../pkg/mod/cache/vcs/0c31a160b38443d5e33ca2a5e60e51734d83293b66dc3d9b0c12699f8d2b5cec for git3 https://github.com/belogik/goes
cd /.../pkg/mod/cache/vcs/0c31a160b38443d5e33ca2a5e60e51734d83293b66dc3d9b0c12699f8d2b5cec; git ls-remote -q origin
# get https://github.com/?go-get=1: 200 OK (0.035s)
0.075s # cd /.../pkg/mod/cache/vcs/0c31a160b38443d5e33ca2a5e60e51734d83293b66dc3d9b0c12699f8d2b5cec; git ls-remote -q origin
# get https://github.com/belogik/goes
# get https://github.com/belogik/goes: 404 Not Found (0.097s)
go get github.com/belogik/goes@latest: module github.com/belogik/goes: git ls-remote -q origin in /.../pkg/mod/cache/vcs/0c31a160b38443d5e33ca2a5e60e51734d83293b66dc3d9b0c12699f8d2b5cec: exit status 128:
	fatal: could not read Username for 'https://github.com': terminal prompts disabled
Confirm the import path was entered correctly.
If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.

This isn't the user experience we want proxy.golang.org to give. I'll work on changing it to successfully return an empty list so that the go command will fall back to @latest.

cuonglm

comment created time in 15 days

issue closedgolang/go

viewing differences in vscode git diff reports "undeclared name" errors for the temp files

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

go version go1.13.5 darwin/amd64

golang.org/x/tools/gopls v0.3.1 golang.org/x/tools/gopls@v0.3.1 h1:yNTWrf4gc4Or0UecjOas5pzOa3BL0WDDyKDV4Wz5VaM=

Does this issue reproduce with the latest release?

not tried yet

What operating system and processor architecture are you using (go env)?

GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/[--]/Library/Caches/go-build" GOENV="/[--]/Library/Application Support/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/[--]/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/[--]/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/54/7pghg1bj18q8kzj3xnvb6d480000gn/T/go-build571485278=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

upgrade gopls to latest version golang.org/x/tools/gopls v0.3.1 golang.org/x/tools/gopls@v0.3.1 h1:yNTWrf4gc4Or0UecjOas5pzOa3BL0WDDyKDV4Wz5VaM=

open a go project in vscode (with go modules)

make some changes to a file

build and test (no errors)

go to git tab and click on a file to see the diff

What did you expect to see?

just the diff of the file being changed. did not expect to see any errors in the problems window

What did you see instead?

was able to see the diff ok.

but noticed that "undeclared name" errors are reported in the PROBLEMS window for a file with name [file-being-diffed].go?path%3d***

the errors do not go away even if i close the diff tab for the file. had to restart vscode

closed time in 15 days

dskdas

issue commentgolang/go

viewing differences in vscode git diff reports "undeclared name" errors for the temp files

Duplicate of #33699. Sorry for the trouble.

dskdas

comment created time in 15 days

issue commentgolang/go

x/tools/gopls: reduce memory usage

@trajber Unfortunately I don't see any signs of a bug in your profile. That doesn't mean there isn't one, but 1.5GiB is not completely out of line for a project with a lot of dependencies, and without access to the source it's very difficult to judge. Sorry, but I don't think there's going to be much we can do in the short term. Hopefully you will benefit from general optimizations we make in the future.

stamblerre

comment created time in 15 days

issue commentgolang/go

x/tools/gopls: high memory usage in sigs.k8s.io/cluster-api

@cbd32 I looked at your profile and I don't see any obvious signs of the autocomplete memory leak, but it'd still be good to rule it out. Once you've reproduced with v0.3.1, please file a new issue so that we can keep separate problems from getting confused with each other.

@vincepri I was also unable to reproduce your issue. I did notice that cluster-api has a number of nested modules, which gopls currently does not support well, but even messing around in those didn't immediately cause a problem. It might be good to pay attention to whether you're working in those when you reproduce it.

vincepri

comment created time in 15 days

issue commentgolang/go

x/tools/gopls: reduce memory usage

@vincepri per the bold section in the original issue, please file a new issue with repro instructions, even if they're just "check this repository out and open it."

stamblerre

comment created time in 15 days

issue commentgolang/go

x/tools/gopls: support something like `compile_commands.json`

I don't think this is feasible in the current architecture. We use the output of go list -json via go/packages to get much more than just a list of files. What are you actually trying to accomplish with this request? Is there some other build tool you want to use? Is it possible to implement a go/packages driver for it?

paulthomson

comment created time in 18 days

IssuesEvent

issue commentgolang/go

go.dev: 404 error should include a link to instructions on how to add a missing module

@natefinch That may be clear to you but it is not clear to me. The repository you linked to is a two year old test, not exactly something I would expect people to need to find in search or read documentation for. Packages with actual usage are highly likely to already be populated. If you think that pkg.go.dev should fetch on demand, feel free to file an issue for that, perhaps with a more realistic example.

@thepudds Adding a pointer to the about page seems reasonable to me. I'll switch this over to that. cc @julieqiu.

thepudds

comment created time in 19 days

issue closedgolang/go

go.dev: 404 error shown for some packages such as github.com/natefinch/markbates

Some packages seem to show a 404 Not Found error, such as:

https://pkg.go.dev/github.com/natefinch/markbates

Sorry for the quick report. Seen via https://twitter.com/NateTheFinch/status/1223427948260347904

CC @natefinch

closed time in 19 days

thepudds

issue commentgolang/go

go.dev: 404 error shown for some packages such as github.com/natefinch/markbates

From https://go.dev/about:

To add a package or module, simply fetch it from proxy.golang.org. Documentation is generated based on Go source code downloaded from the proxy.golang.org/<module>@<version>.zip. New module versions are fetched from index.golang.org and added to the go.dev site every few minutes.

Nobody had ever downloaded it. It's on pkg.go.dev now.

thepudds

comment created time in 19 days

issue commentgolang/go

proxy.golang.org: Enable CORS?

Since everything on proxy/sum/index.golang.org is a simple GET request that could be performed by an iframe just as easily, this seems safe in my limited understanding. Still, it's work, so it'd be nice to have some example use cases to justify it.

marwan-at-work

comment created time in 22 days

issue commentgolang/go

go/build: doesn't properly use backslashes on Windows

IsLocalImport reports whether the import path is...

Import paths are slash separated. The upstream project shouldn't be using IsLocalImport on filesystem paths.

al45tair

comment created time in 22 days

issue commentgolang/go

x/tools/gopls: format_on_save_new_file (govim) test flaky

https://github.com/golang/go/issues/36772#issuecomment-579405215: Okay. I don't understand how that's possible; I'll need another log to investigate further.

myitcv

comment created time in 24 days

issue commentgolang/go

x/tools/gopls: unexpected errors logged in various situations

Did my CL not fix (2)?

myitcv

comment created time in 24 days

issue commentgolang/go

x/tools/gopls: misleading diagnostic for badly formed func literal

This would be coming from go/parser or something, no? I don't think gopls can do anything to improve it.

myitcv

comment created time in a month

issue commentgolang/go

x/tools/gopls: "could not import ___" after adding new _test package

I believe the problem is that packages.Load("./...") does not return the overlay packages.

zikaeroh

comment created time in a month

issue commentgolang/go

x/tools/gopls: go directive added to go.mod with tempModfile = true

With gopls from master and Go 1.14, the test gets past the cmp and fails later:

        # Verify that there have been no changes to go.mod (0.000s)
        > cmp go.mod go.mod.golden
        # Verify that we have no diagnostics for any files (25.766s)
        > vimexprwait errors.empty getqflist()
        [stderr]
        -[]
        [exit status 1]
        FAIL: testdata/scenario_tempmodfile/modfile_changes.txt:18: unexpected command failure

Dunno what that means but I would think this bug, at least, is fixed.

myitcv

comment created time in a month

issue commentgo-delve/delve

'regs' does not show selected frame registers.

I'd think this would be generally useful for panics, especially in optimized binaries with location lists.

I think @aarzilli pointed to this, but to be explicit: It should be possible to recover the registers in at least some circumstances. On Linux it'd be the ctxt argument to sigtrampgo: https://github.com/golang/go/blob/67f0f83216930e053441500e2b28c3fa2b667581/src/runtime/signal_unix.go#L403 On Windows it looks like https://github.com/golang/go/blob/6dbcc8b8651909442ff823231daba096f447a163/src/runtime/signal_windows.go#L104 messes with things badly enough that it would be hard to do.

It seems easy enough to add a sigctxt field somewhere (maybe the m?) and have all the OS handlers fill that in with whatever the platform-specific context structure is. Please feel free to file an issue if that'd help.

klauspost

comment created time in a month

issue commentgolang/go

x/tools/gopls: diagnostics not received for new files

The problem here is that the initial workspace load finds nothing, and AFAIK we currently have no mechanism for adding packages to the workspace set, so they don't ever end up getting diagnosed in the background.

myitcv

comment created time in a month

issue commentgolang/go

x/tools/gopls: race in updateAnalyzers

I set GORACE=history_size=7 and had no luck getting the other side of the race.

stamblerre

comment created time in a month

issue commentgolang/go

x/tools/gopls: cache results of packages.Load

I think this is obsoleted by the new whole-workspace load strategy.

stamblerre

comment created time in a month

issue closedgolang/go

x/tools/gopls: slow imports / formatting

@zikaeroh reported slow import organization and formatting in https://github.com/golang/go/issues/35388. Filing this issue to follow up. If anyone else experiences similar issues, please comment below and attach gopls logs if possible.

closed time in a month

stamblerre

issue commentgolang/go

x/tools/gopls: slow imports / formatting

Meh, I'm gonna close this and we can reopen/refile if it actually shows up.

stamblerre

comment created time in a month

issue commentgolang/go

x/tools/gopls: "reply not invoked with a valid call" error showing up

Thanks for the initial diagnosis, I'd noticed this but not gotten around to investigating.

muirdm

comment created time in a month

issue commentgolang/go

godoc.org: shows wrong import path for forked Go projects

I don't see a bug here. The first quote is the correct import path for the fork. The second quote is from the README, which was forked unmodified from upstream. If anything is wrong, it would seem to be the README.

mcandre

comment created time in a month

issue commentgolang/go

godoc.org: various degrees of service degradation and unavailability on Sunday, January 19, 2020

To my knowledge, nobody has ever committed to an availability goal for godoc.org. It is operated as a best-effort service, and it is slated to be replaced by pkg.go.dev. @dmitshur took time out of his weekend to fix it, which as far as I'm concerned was beyond the call of duty. I think it would have been nice to at least thank him. I don't think it's fair to assume that a post mortem is owed.

This is quite different from other services, notably proxy/index/sum.golang.org, which are critical dependencies for many Go users. We don't have a published policy for post mortems, but I think if those experienced an outage anything like this scale it would be appropriate to publish one.

If anyone would like to discuss the level of support that these services receive, I think a thread on golang-dev or golang-nuts would be a better forum. Or, if there's a concrete proposal, (e.g. publicize a post mortem for any significant outage) perhaps file an issue. But I don't think this discussion is shaping up to be productive.

heschik

comment created time in a month

issue openedgolang/go

godoc.org: is down

All pages are returning Internal server error.

cc @dmitshur @andybons and maybe @bradfitz ?

created time in a month

issue commentgolang/go

x/tools/gopls: slow imports / formatting

@zikaeroh I believe we discussed on Slack that this had gotten much better. Close?

stamblerre

comment created time in a month

issue closedgolang/go

x/tools/gopls: fuzzy matching in unimported completion interferes with "good" (prefix/substring) choices

Please answer these questions before submitting your issue. Thanks!

What did you do?

In my project (https://github.com/hortbot/hortbot, relatively large overall dependency set), add a new test file with no imports, then make a new test and complete on unimported packages.

What did you expect to see?

Typing something like errors.New or assert.NilError gives me a resonable set of choices. I.e. I want to see errors.New, or pkg/errors.New, etc, at the top as they are perfect matches.

What did you see instead?

For errors.New, I get New from a few packages, then a bunch of other results I don't care about that contain the letters "NEW" in order (NewResourceCreationErrorEnum, LabelErrorEnum_CANNOT_ATTACH_NON_MANAGER_LABEL_TO_CUSTOMER), then more exact New matches, and them more fuzzy matches, and so on.

Logs: https://gist.github.com/zikaeroh/385ca2756bffd1af9ab49e199c526a2b

Build info

golang.org/x/tools/gopls master
    golang.org/x/tools/gopls@v0.1.8-0.20200116011002-7042ee646e79 h1:DR0yHhyXch/edNKuX31OXiMcMp1SLwPZnQOx3WwNprI=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
    golang.org/x/tools@v0.0.0-20200116011002-7042ee646e79 h1:vUYHqXKakw93ZB7hlBqmqr5mOVwQaMWS38qZgBHpdIQ=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info

go version go1.13.6 linux/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jake/.cache/go-build"
GOENV="/home/jake/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jake/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jake/zikaeroh/hortbot/hortbot/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build966422196=/tmp/go-build -gno-record-gcc-switches"

closed time in a month

zikaeroh

issue commentgolang/go

x/tools/gopls: fuzzy matching in unimported completion interferes with "good" (prefix/substring) choices

Hopefully this is fixed enough for v0.3. @muirdm, do you think we need to make the changes you described above, or just keep them in mind in case of future issues? I don't fully understand the implications.

zikaeroh

comment created time in a month

issue commentgolang/go

x/tools/gopls: fuzzy matching in unimported completion interferes with "good" (prefix/substring) choices

We could make it only go down, but then the caller of found() would need to call matchingCandidate() to set the starting score.

I don't think that's true? Anywhere we multiply by 10 today, the caller could pass score*10 and we could divide by 10 everywhere we aren't multiplying now. Just so that it's more predictable what the score range of any given call could be.

I reviewed all the callers of matchingCandidate, and it looks like it will be harmless to return false instead of true -- it controls stuff like creating composite literals and taking references, none of which makes sense if you don't know the type. That resolves the immediate complaint in this bug. CL shortly.

zikaeroh

comment created time in a month

issue commentgolang/go

cmd/go: invalid version: unknown revision when GOPROXY/GOSUMDB disabled

FYI @hyangah @katiehockman.

It would be interesting to say what the Fedora people have to say about this. I expect it will only happen more often as time goes on and more things are deleted.

carlpett

comment created time in a month

issue commentgolang/go

x/tools/gopls: fuzzy matching in unimported completion interferes with "good" (prefix/substring) choices

https://github.com/golang/tools/blob/6edc0a871e694a1c8728996c84668863c13d2b4f/internal/lsp/source/completion.go#L326

maybe? I think we should have a rule that scores only go one direction, probably down, from the ones passed in.

zikaeroh

comment created time in a month

issue commentgolang/go

x/tools/gopls: fuzzy matching in unimported completion interferes with "good" (prefix/substring) choices

You're right, something is giving the worse candidates a much higher score than I set by a factor of 10. All my scores are .01 * 7 at most.

source.CompletionItem{Label:"AdErrorEnum_CANNOT_SET_FIELD_WITH_AD_ID_SET_FOR_SHARING", Detail:"(from \"google.golang.org/genproto/googleapis/ads/googleads/v1/errors\")", InsertText:"AdErrorEnum_CANNOT_SET_FIELD_WITH_AD_ID_SET_FOR_SHARING", Kind:var, AdditionalTextEdits:[]protocol.TextEdit{protocol.TextEdit{Range:1:0-1:0, NewText:"\nimport \"google.golang.org/genproto/googleapis/ads/googleads/v1/errors\"\n"}}, Depth:0, Score:0.4, snippet:(*snippet.Builder)(nil), Documentation:""} vs source.CompletionItem{Label:"As", Detail:"func(err error, target interface{}) bool (from \"errors\")", InsertText:"As", Kind:func, AdditionalTextEdits:[]protocol.TextEdit{protocol.TextEdit{Range:1:0-1:0, NewText:"\nimport \"errors\"\n"}}, Depth:0, Score:0.07, snippet:(*snippet.Builder)(0xc011946cc0), Documentation:"As finds the first error in err's chain that matches target, and if so, sets target to that error value and returns true."}

zikaeroh

comment created time in a month

issue closedgolang/go

x/tools/gopls: >45s for first completion of unimported package

<!-- Please answer these questions before submitting your issue. Thanks! -->

What version of Go are you using (go version)?

<pre> $ go version go version go1.13.5 darwin/amd64 $ go list -m golang.org/x/tools golang.org/x/tools v0.0.0-20191219041853-979b82bfef62 $ go list -m golang.org/x/tools/gopls golang.org/x/tools/gopls v0.1.8-0.20191219041853-979b82bfef62 </pre>

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/Users/pontus/Library/Caches/go-build" GOENV="/Users/pontus/Library/Application Support/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/pontus/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/Users/pontus/proj/govim/latest_tools/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xm/ls208xd174v95pgd20rn_qbh0000gn/T/go-build213935326=/tmp/go-build -gno-record-gcc-switches -fno-common" </pre></details>

What did you do?

Open up a new module

$ cd $(mktemp -d) && go mod init x && vi main.go

Then type in:

package main

func main() {
        fmt.|
}

And at | trigger a completion request. <!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

What did you expect to see?

Completion request return a little bit faster.

What did you see instead?

The response arrives after ~47 seconds. See https://gist.github.com/leitzler/29eca825af07416bae63c1c304527994#file-gopls-log-L429

As per discussion on Slack with @heschik it looks like it is due to package name loading.

Rerun with verboseOutput:

2019/12/19 21:43:19 gopathwalk: scanning /Users/pontus/go/pkg/mod
2019/12/19 21:43:19 Directory added to ignore list: /Users/pontus/go/pkg/mod/cache
2019/12/19 21:43:23 gopathwalk: scanned /Users/pontus/go/pkg/mod in 3.864949127s
[Trace - 21:44:04.398 PM] Received response 'textDocument/completion - (2)' in 44771ms.

Mod cache size:

$ time find $(go env GOPATH)/pkg/mod/ -type d | wc -l
   79000
real	0m10.360s
user	0m0.519s
sys	0m6.519s

closed time in a month

leitzler

issue commentgolang/go

x/tools/gopls: >45s for first completion of unimported package

This should be fixed, or at least fixed enough to be on by default.

leitzler

comment created time in a month

issue commentgolang/go

x/tools/gopls: go list error opening Go source tree

This might be https://golang.org/issue/36188. Are you absolutely sure that gopls is using the go binary that matches the GOROOT you're opening?

myitcv

comment created time in a month

issue commentgolang/go

x/tools/gopls: fuzzy matching in unimported completion interferes with "good" (prefix/substring) choices

cc @muirmanders for fuzzy matching issues, but I'm confident there are unimported completion scoring issues in here too so I'll take it for now.

zikaeroh

comment created time in a month

issue commentgolang/go

x/tools/gopls: go list error opening Go source tree

OK that was a me problem. Works for me. @myitcv, wanna take another look?

myitcv

comment created time in a month

issue commentgolang/go

cmd/go: panics listing ./... in GOROOT/src

I have a habit of leaving toy .go files in $GOROOT/src, and apparently that's a bad idea. Definitely not a release blocker.

heschik

comment created time in a month

issue commentgolang/go

x/tools/gopls: go list error opening Go source tree

I tried again and hit #36587...

myitcv

comment created time in a month

issue openedgolang/go

cmd/go: panics listing ./... in GOROOT/src

What version of Go are you using (go version)?

<pre> $ ~/go/bin/go version
go version devel +f77e7ed7e3 Wed Jan 15 22:02:44 2020 +0000 linux/amd64 </pre>

Does this issue reproduce with the latest release?

Yes, 1.13.4.

What did you do?

cd $(go env GOROOT); go list ./...

What did you expect to see?

Some packages

What did you see instead?

panic: loadPackageData called with empty package path

goroutine 33 [running]:
cmd/go/internal/load.loadPackageData(0x0, 0x0, 0x0, 0x0, 0xc000026034, 0x24, 0x0, 0x0, 0x0, 0x0, ...)
	go/src/cmd/go/internal/load/pkg.go:649 +0x625
cmd/go/internal/load.(*preload).preloadMatches.func1(0xc0004aeba0, 0x0, 0x0)
	go/src/cmd/go/internal/load/pkg.go:847 +0x80
created by cmd/go/internal/load.(*preload).preloadMatches
	go/src/cmd/go/internal/load/pkg.go:845 +0x165

@jayconrod @bcmills, marking as a tentative release blocker since it breaks gopls on the stdlib.

created time in a month

issue closedgolang/go

x/tools/gopls: limit number of returned completions (maybe just unimported)

With unimported completions on, triggering completions with no prefix is very slow. Depressingly, the main problem is generating the diffs for all the imports that could be added -- after https://golang.org/cl/205678, we need to do some fairly expensive parsing and formatting for each one. Profile:

profile001

There's really no point in returning every single package in the module cache as a completion candidate. I think we should definitely limit those.

I think @muirdm has also said that Emacs may have trouble handling large numbers of completions regardless. So maybe we should just put a cap on the entire result set?

closed time in a month

heschik

issue commentgolang/go

x/tools/gopls: limit number of returned completions (maybe just unimported)

This was done in the commits above.

heschik

comment created time in a month

IssuesEvent

issue closedgolang/go

x/tools/gopls: >45s for first completion of unimported package

<!-- Please answer these questions before submitting your issue. Thanks! -->

What version of Go are you using (go version)?

<pre> $ go version go version go1.13.5 darwin/amd64 $ go list -m golang.org/x/tools golang.org/x/tools v0.0.0-20191219041853-979b82bfef62 $ go list -m golang.org/x/tools/gopls golang.org/x/tools/gopls v0.1.8-0.20191219041853-979b82bfef62 </pre>

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/Users/pontus/Library/Caches/go-build" GOENV="/Users/pontus/Library/Application Support/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/pontus/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/Users/pontus/proj/govim/latest_tools/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xm/ls208xd174v95pgd20rn_qbh0000gn/T/go-build213935326=/tmp/go-build -gno-record-gcc-switches -fno-common" </pre></details>

What did you do?

Open up a new module

$ cd $(mktemp -d) && go mod init x && vi main.go

Then type in:

package main

func main() {
        fmt.|
}

And at | trigger a completion request. <!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

What did you expect to see?

Completion request return a little bit faster.

What did you see instead?

The response arrives after ~47 seconds. See https://gist.github.com/leitzler/29eca825af07416bae63c1c304527994#file-gopls-log-L429

As per discussion on Slack with @heschik it looks like it is due to package name loading.

Rerun with verboseOutput:

2019/12/19 21:43:19 gopathwalk: scanning /Users/pontus/go/pkg/mod
2019/12/19 21:43:19 Directory added to ignore list: /Users/pontus/go/pkg/mod/cache
2019/12/19 21:43:23 gopathwalk: scanned /Users/pontus/go/pkg/mod in 3.864949127s
[Trace - 21:44:04.398 PM] Received response 'textDocument/completion - (2)' in 44771ms.

Mod cache size:

$ time find $(go env GOPATH)/pkg/mod/ -type d | wc -l
   79000
real	0m10.360s
user	0m0.519s
sys	0m6.519s

closed time in a month

leitzler

issue commentgolang/go

x/tools/gopls: >45s for first completion of unimported package

Since I keep getting distracted, one more update: https://golang.org/cl/212022 is the last thing I intend to do here. Other than that we should be in good shape.

leitzler

comment created time in a month

issue commentgolang/go

x/tools/gopls: add a custom command to ask the server if diagnostics are all sent

The CL above is primarily to make the command line client sane in the case where a file doesn't have diagnostics published for whatever reason. I don't really have an opinion on how tests should work in general, I'll let you two discuss that :)

stamblerre

comment created time in a month

issue closedgolang/go

x/tools/gopls: pre-warm the cache for imports

Original conversation in https://github.com/golang/go/issues/32750#issuecomment-523141204.

closed time in a month

stamblerre

issue commentgolang/go

x/tools/gopls: pre-warm the cache for imports

Not really, but it's probably moot now. Closing in favor of all the stuff I did for https://golang.org/issue/36222.

stamblerre

comment created time in a month

issue commentgolang/go

cmd/link: bad debug_frame section on macOS with buildmode=pie [Debugging]

@jeremyfaller I think you would be first stop for this stuff now? Especially if you have a Mac handy -- I let mine rot.

aarzilli

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: >45s for first completion of unimported package

Submitted a bunch of stuff. Most everything from the above is done. Remaining:

  • Detachable walks. In this report, it took 4 seconds to do the filesystem walk. We shouldn't hang the request that long.
  • Reattachable walks. Following the above, it would be spiffy to reattach to a walk in progress, rather than blocking for it to finish.
  • Incremental rescans. I'm on the fence about this. With lazy package loading, it might be sufficient to do the walk in the background (see https://golang.org/cl/212022) and then let the user pull in more stuff.
leitzler

comment created time in 2 months

issue commentgolang/go

runtime: pclntab grows superlinearly

I think there's some confusion in the blog post. Sorry if I'm the one confused instead.

The blog post talks about runtime.pclntab, but that symbol actually contains many tables, correct? ld.(*Link).pclntab() writes the pcfile and pcline tables, which are roughly in line with what you'd expect to be in a symbol named pclntab, but it also writes the pcsp and pcdata tables, which are quite different.

Has anyone checked which tables are actually the problem? I think all the commentary above is discussing the file/line tables.

robpike

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: >45s for first completion of unimported package

Notes to self as of https://golang.org/cl/212634:

  • Missed a major trick in the discussion above, which is simply to stop doing work when it looks like we have enough results to overwhelm the user. That's done.
  • Scan cancellation is almost done. gopathwalk cancellation is impossible without redesigning fastwalk to be postorder, but we can detach them easily enough.
    • It might make sense to process the cache first to see if that yields enough results, and only then do the walks. But then we need to know what's new, vs. rescanning the whole cache.
  • The API now exists to do goimports-style directory filtering, just have to implement it.
leitzler

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: improve support for authoring cgo packages

This is an experimental patch set. Please be ready for some glitches.

Okay, here you go. Note that the patched Go installation must be first on your path when you run your editor, otherwise it won't work.

While trying it out, please use this mental model: The "C" package is populated on demand based on your cgo comments and the identifiers used by your Go code. It will generally only be created once when gopls starts up. If you modify a cgo comment, or start using a new identifier, you may have problems with things not being found, or not being of the right type. In that case you'll need to restart the language server. In VS Code, that means pressing ctrl-shift-P and choosing "Go: Restart Language Server".

Other than that, features should all just work. For now, let's keep feedback on this issue. I may ask you to file new ones if it gets messy.

$ mkdir ~/gopls-cgo
$ cd ~/gopls-cgo
$ git clone https://go.googlesource.com/go
Cloning into 'go'...
remote: Counting objects: 22, done59 MiB ...Counting objects: 1   
remote: Total 403135 (delta 306410), reused 403135 (delta 306410)
Receiving objects: 100% (403135/403135), 260.03 MiB | 39.93 MiB/s, done.
Resolving deltas: 100% (306410/306410), done.
$ cd go/src
$ git checkout -b cgo
$ git fetch "https://go.googlesource.com/go" refs/changes/77/33677/7 && git cherry-pick FETCH_HEAD
From https://go.googlesource.com/go
 * branch                  refs/changes/77/33677/7 -> FETCH_HEAD
Auto-merging src/go/types/resolver.go
Auto-merging src/go/types/check.go
[...]
$ GOROOT_BOOTSTRAP=$(dirname $(which go))/.. ./make.bash
[...]
$ cd ~/gopls-cgo 
$ export PATH=$HOME/gopls-cgo/go/bin:$PATH
$ git clone https://go.googlesource.com/tools                                                     
Cloning into 'tools'...
remote: Counting objects: 16, done0 MiB ...Counting objects: 1   
remote: Total 36491 (delta 24075), reused 36491 (delta 24075)
Receiving objects: 100% (36491/36491), 18.77 MiB | 39.80 MiB/s, done.
Resolving deltas: 100% (24075/24075), done.
$ cd tools/gopls
$ git checkout -b cgo
$ git fetch "https://go.googlesource.com/tools" refs/changes/64/208264/1 && git cherry-pick FETCH_HEAD
From https://go.googlesource.com/tools
 * branch              refs/changes/64/208264/1 -> FETCH_HEAD
Auto-merging go/packages/packages.go
Auto-merging go/packages/golist.go
[...]
$ go install
heschik

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: >45s for first completion of unimported package

I'm going to hijack this for a braindump of performance problems with unimported completions.

  • With a large enough GOPATH/module cache, a scan can be prohibitively expensive. We cannot block a user action on a scan finishing.

    • This implies either cancelling the scan after the completion budget expires, or detaching it to run asynchronously.
    • It also makes dumping the whole cache and rescanning it from scratch a bad idea. We need to update incrementally or do the rescan in the background (https://golang.org/cl/212022).
    • Ideally we would do the scan in relevancy order so that we can give the most useful completions as fast as possible. In module mode, at least, we can prime the cache based on the go list -m results, then do the full scan later. In GOPATH mode I don't think there's much to be done since the only signal is imported packages and we should have those anyway, see below.
  • In GOPATH mode, anything can change at any time. We have to trade off between latency, or at least resource consumption, and correctness.

    • A mitigating factor is that we prefer to use real type-check results when available, which means we'll have correct data for stuff already in workspace scope no matter what. Small bug here: the outer loop is only over the things found in the scan, so if there's a candidate known to gopls but not in the scan we'll miss it.
  • In module mode, we can assume that module cache entries don't change. All we need to do is add new things, and maybe delete everything if the user runs clean -modcache.

  • Since GOPATH and the module cache can change so differently, there isn't a single brute-force strategy that works for both. Ideally we can update the cache incrementally, which is a good strategy in both cases.

    • Incremental updates are also good for user experience; we can give them the best results we have and continue re-scanning in the background.
  • No matter the strategy, scans chew up CPU time/power/disk bandwidth. I don't want to add knobs, but I also don't want to be doing 47 seconds of work every 30 seconds...

  • At least in this report, loading package names was the most expensive part of the process by far. We could load names on demand based on a heuristic match of the filesystem path, as goimports does today.

@stamblerre @muirmanders @ianthehat

leitzler

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: normalize feature sets of unimported completions and organize imports

...except that addExternalCandidates goes to great lengths to return the first usable candidate it finds for speed reasons. It would be very nice to prioritize the scan in the right order, rather than sorting afterward.

SteelPhase

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: normalize feature sets of unimported completions and organize imports

Retitling to reflect the remaining work.

Unimported completions are missing the "sibling imports" trick. This would be a nice feature. I'm not sure whether to reuse the implementation from the imports package or rewrite it from scratch. Sharing code would be nice, but gopls already has all the parsing done. Just pass all the sibling files to GetPackageExports?

Organize Imports is missing the sorting by relevance. This should be easy to plug in to addExternalCandidates, as long as we want it to affect goimports. Sibling imports will take precedence, but that's probably appropriate.

SteelPhase

comment created time in 2 months

issue commentgolang/go

cmd/go: 'go list' should set the 'GoMod' field to reflect -modfile

goimports and gopls should be unaffected by this either way.

bcmills

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: slow imports / formatting

Maybe not, but in the past I had been getting format + import all in one go. Since the race condition was fixed, I haven't seen it happen in one step anymore.

Right. There were always technically two calls, but imports would do formatting so yeah.

I believe there's an open issue about sorting those better (i.e. sorting direct module dependencies first), which would help somewhat.

#36077. Mailed https://golang.org/cl/212021.

If I accidentally select the wrong one, it's a pain to undo because go.mod will have already changed.

For sure. That's https://golang.org/cl/211538, but will require 1.14.

stamblerre

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: slow imports / formatting

I can just bulk record logs as I work on something, and hope to find something, but know that I definitely won't be able to do any sort of screencasting.

That's totally fine -- the log is sufficient. I don't really need the screencasts at all for this, I should be able to piece things together from the logs if I absolutely have to.

It's the two together (i.e. goimports-style formatting) where I see slowness.

Yeah, I guess what I meant was that if you're doing the kind of stuff in the screencast, changing imports very fast, then I'm not surprised if gopls starts tripping over itself. A less synthetic example would help make sure I'm focused on the right problem.

I find myself having to wait / save again after imports get sorted to get things to the right state.

Maybe this is the actual problem. Would you be so worried about the performance if you didn't have to do things twice? I would be interested to see a log of formatting not happening, maybe as a separate bug.

FWIW, with completeUnimported all this should matter less, because the import statement will show up in the right place before you save.

stamblerre

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: slow imports / formatting

One note, actually -- changing the imports block like this is a worst case for gopls because it triggers Load calls. If you can get this kind of thing to happen by changing non-imports, that's much more worrying/interesting. If it only happens in imports, then I'm tempted to say that people are unlikely to add imports multiple times per second.

stamblerre

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: slow imports / formatting

Sorry for the delay on this, last week was hectic. Needless to say, I still can't reproduce the problem...

Not sure what to make of this yet. I don't think it has all that much to do with imports per se. Request 48, for example, takes 694ms. But almost all of that time is actually just spent waiting for a previous request to finish a go/packages.Load call.

<details>

[Trace - 21:54:14.256 PM] Sending request 'textDocument/codeAction - (48)'.
Params: {"textDocument":{"uri":"file:///home/jake/zikaeroh/hortbot/hortbot/internal/cli/flags/botflags/botargs.go"},"range":{"start":{"line":21,"character":0},"end":{"line":21,"character":0}},"context":{"diagno>


[Trace - 21:54:14.300 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 42.640563ms for GOROOT=/usr/lib/go GOPATH=/home/jake/go GO111MODULE= PWD=/home/jake/zikaeroh/hortbot/hortbot go \"env\" \"GOMOD\", stderr: \u003c\u003c\u003e\u00>


[Trace - 21:54:14.437 PM] Sending request 'textDocument/documentLink - (49)'.
Params: {"textDocument":{"uri":"file:///home/jake/zikaeroh/hortbot/hortbot/internal/cli/flags/botflags/botargs.go"}}


[Trace - 21:54:14.516 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 245.449982ms for GOROOT=/usr/lib/go GOPATH=/home/jake/go GO111MODULE= PWD=/home/jake/zikaeroh/hortbot/hortbot go \"list\" \"-m\" \"-json\" \"all\", stderr: \u003>


[Trace - 21:54:14.729 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 609.25391ms for GOROOT=/usr/lib/go GOPATH=/home/jake/go GO111MODULE= PWD=/home/jake/zikaeroh/hortbot/hortbot go \"list\" \"-e\" \"-json\" \"-compiled=true\" \"-t>


[Trace - 21:54:14.731 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 go/packages.Load\n\tpackages = 1"}


[Trace - 21:54:14.731 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 go/packages.Load\n\tpackage = github.com/hortbot/hortbot/internal/cli/flags/botflags\n\tfiles = [/home/jake/zikaeroh/hortbot/hortbot/internal/cli/flags/botflags/>


[Trace - 21:54:14.732 PM] Received notification 'textDocument/publishDiagnostics'.
Params: {"uri":"file:///home/jake/zikaeroh/hortbot/hortbot/internal/cli/flags/botflags/botargs.go","version":11,"diagnostics":[]}


[Trace - 21:54:14.949 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 go/packages.Load\n\tpackages = 1"}


[Trace - 21:54:14.949 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 go/packages.Load\n\tpackage = github.com/hortbot/hortbot/internal/cli/flags/botflags\n\tfiles = [/home/jake/zikaeroh/hortbot/hortbot/internal/cli/flags/botflags/>


[Trace - 21:54:14.950 PM] Received notification 'window/logMessage'.
Params: {"type":3,"message":"2019/12/06 21:54:14 fixImports(filename=\"/home/jake/zikaeroh/hortbot/hortbot/internal/cli/flags/botflags/botargs.go\"), abs=\"/home/jake/zikaeroh/hortbot/hortbot/internal/cli/flags>


[Trace - 21:54:14.951 PM] Received response 'textDocument/codeAction - (48)' in 694ms.
Result: {}

</details>

Request 19 is also slow, in the actual goimports code this time, but that's the first scan before anything is cached, so I don't think there's much to be learned there.

Sorry, I think I'm going to need more logs to nail this down. If you can give a snippet from a natural occurrence, rather than a forced repro, that would be ideal. But I understand that might not be convenient, so whatever you can do.

stamblerre

comment created time in 2 months

issue commentgolang/go

gopls panic: runtime error: slice bounds out of range [42:40]

I am really not sure how this can happen. Maybe something strange with junction points or symlinks?

Can you add an explicit panic in that code that shows dir and root.Path?

try876

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: new import not loaded until file is saved

That is probably happening in the go/packages overlay support, which needs to augment the response with the packages that are imported only in the overlay.

Sigh.

ian-mi

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: new import not loaded until file is saved

There's also some code path where we do go list on an import path, which also doesn't work; afaict you have to list the vendor path, e.g. github.com/google/knative-gcp/vendor/github.com/google/go-cmp/cmp.

ian-mi

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: new import not loaded until file is saved

At a bare minimum, this code simply won't work with GOPATH vendoring. https://github.com/golang/tools/blob/210e553fe1f64b5f29af9b359e51cb5216e1cd8e/internal/lsp/cache/check.go#L312

ian-mi

comment created time in 2 months

issue commentgolang/go

x/tools/gopls: new import not loaded until file is saved

Your logs weirdly don't mention cmp at all, but I was able to reproduce the problem by checking out knative-gcp and following the same steps.

Pretty sure this has something to do with vendoring. I get errors like

[Error - 3:46:18 PM] 2019/12/17 15:46:18 no dep handle: no metadata for github.com/google/knative-gcp/vendor/github.com/google/go-cmp/cmp
        package = github.com/google/knative-gcp/vendor/github.com/google/go-cmp/cmp

which are quite suspicious.

ian-mi

comment created time in 2 months

issue commentgolang/go

verifying module@v0.9.0/go.mod: ... reading module@v0.9.0: 410 Gone

410 is necessarily for internal implementation reasons.

The need for negative caching is described in https://github.com/golang/go/issues/34370#issuecomment-532784418.

mholt

comment created time in 2 months

issue commentgolang/go

verifying module@v0.9.0/go.mod: ... reading module@v0.9.0: 410 Gone

I checked the internal logs for proxy/sum.golang.org. The first request they ever got for certmagic v0.9.0 was at 2019-12-17 17:23:35 UTC, which is the request that resulted in the negative cache entry in question. The GitHub UI for https://github.com/mholt/certmagic/releases/tag/v0.9.0 says that it was tagged at 12:22 EST, or 17:22 UTC. I don't have any explanation for the mismatch; the go command just says that it couldn't find that version. Was the tag created using the GitHub UI, in which case maybe there's some delay there?

mholt

comment created time in 2 months

more