Thanks for the report. I'll take a look.

Could you please attach "go env" output from the virtualbox environment?

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

<pre> $ go version $ go version go version go1.15.2 linux/amd64 $ gopls version golang.org/x/tools/gopls v0.5.0 golang.org/x/tools/[email protected] h1:XEmO9RylgmaXp33iGrWfCGopVYDGBmLy+KmsIsfIo8Y= </pre>

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/usr/local/google/home/thanm/.cache/go-build" GOENV="/usr/local/google/home/thanm/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/tmp/gopath/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/tmp/gopath" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/tmp/go.repo" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/tmp/go.repo/pkg/tool/linux_amd64" GCCGO="/ssd/gcc-trunk/cross/bin/gccgo" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="/tmp/go.repo/src/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build118750431=/tmp/go-build -gno-record-gcc-switches"

</pre></details>

What did you do?

I am trying to use vscode to develop the Go standard library, as described in https://github.com/golang/vscode-go/blob/master/docs/stdlib.md. While I am able to get vscode up and running, I can't seem to get the correct set of cross references for exported package symbols.

I've attached a small shell script that checks out and builds Go stdlib and gopls, then invokes vscode in the proper spot. In the vscode session, I navigate to <goroot>/src/cmd/internal/goobj2/objfile.go, then select the symbol PkgIdxSelf on line 142, and then select "Find All References".

What did you expect to see?

A similar operation using "guru" returns this set:

/tmp/go.repo/src/cmd/internal/goobj2/objfile.go:143.2-143.11: references to const PkgIdxSelf untyped int
/tmp/go.repo/src/cmd/internal/goobj/readnew.go:63.15-63.24: case goobj2.PkgIdxSelf:
/tmp/go.repo/src/cmd/internal/goobj/readnew.go:131.33-131.42: if a.Sym().PkgIdx != goobj2.PkgIdxSelf {
/tmp/go.repo/src/cmd/internal/obj/objfile2.go:336.56-336.65: case goobj2.PkgIdxNone, goobj2.PkgIdxBuiltin, goobj2.PkgIdxSelf: // not an external indexed reference
/tmp/go.repo/src/cmd/internal/obj/objfile2.go:437.19-437.28: PkgIdx: goobj2.PkgIdxSelf,
/tmp/go.repo/src/cmd/internal/obj/objfile2.go:452.22-452.31: s.PkgIdx = goobj2.PkgIdxSelf
/tmp/go.repo/src/cmd/internal/obj/sym.go:197.22-197.31: s.PkgIdx = goobj2.PkgIdxSelf
/tmp/go.repo/src/cmd/link/internal/loader/loader.go:600.14-600.23: case goobj2.PkgIdxSelf:
/tmp/go.repo/src/cmd/compile/internal/gc/iexport.go:1002.27-1002.36: if lsym.PkgIdx > goobj2.PkgIdxSelf || (lsym.PkgIdx == goobj2.PkgIdxInvalid && !lsym.Indexed()) || s.Linkname != "" {

What did you see instead?

Only returns references in the source file (objfile.go).

vscode-xref-repro-shellscript.txt

Thanks Heschi -- makes sense. Appreciate the help; I'll close out the bug.

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

<pre> $ go version $ go version go version go1.15.2 linux/amd64 $ gopls version golang.org/x/tools/gopls v0.5.0 golang.org/x/tools/[email protected] h1:XEmO9RylgmaXp33iGrWfCGopVYDGBmLy+KmsIsfIo8Y= </pre>

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/usr/local/google/home/thanm/.cache/go-build" GOENV="/usr/local/google/home/thanm/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/tmp/gopath/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/tmp/gopath" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/tmp/go.repo" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/tmp/go.repo/pkg/tool/linux_amd64" GCCGO="/ssd/gcc-trunk/cross/bin/gccgo" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="/tmp/go.repo/src/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build118750431=/tmp/go-build -gno-record-gcc-switches"

</pre></details>

What did you do?

I am trying to use vscode to develop the Go standard library, as described in https://github.com/golang/vscode-go/blob/master/docs/stdlib.md. While I am able to get vscode up and running, I can't seem to get the correct set of cross references for exported package symbols.

I've attached a small shell script that checks out and builds Go stdlib and gopls, then invokes vscode in the proper spot. In the vscode session, I navigate to <goroot>/src/cmd/internal/goobj2/objfile.go, then select the symbol PkgIdxSelf on line 142, and then select "Find All References".

What did you expect to see?

A similar operation using "guru" returns this set:

/tmp/go.repo/src/cmd/internal/goobj2/objfile.go:143.2-143.11: references to const PkgIdxSelf untyped int
/tmp/go.repo/src/cmd/internal/goobj/readnew.go:63.15-63.24: case goobj2.PkgIdxSelf:
/tmp/go.repo/src/cmd/internal/goobj/readnew.go:131.33-131.42: if a.Sym().PkgIdx != goobj2.PkgIdxSelf {
/tmp/go.repo/src/cmd/internal/obj/objfile2.go:336.56-336.65: case goobj2.PkgIdxNone, goobj2.PkgIdxBuiltin, goobj2.PkgIdxSelf: // not an external indexed reference
/tmp/go.repo/src/cmd/internal/obj/objfile2.go:437.19-437.28: PkgIdx: goobj2.PkgIdxSelf,
/tmp/go.repo/src/cmd/internal/obj/objfile2.go:452.22-452.31: s.PkgIdx = goobj2.PkgIdxSelf
/tmp/go.repo/src/cmd/internal/obj/sym.go:197.22-197.31: s.PkgIdx = goobj2.PkgIdxSelf
/tmp/go.repo/src/cmd/link/internal/loader/loader.go:600.14-600.23: case goobj2.PkgIdxSelf:
/tmp/go.repo/src/cmd/compile/internal/gc/iexport.go:1002.27-1002.36: if lsym.PkgIdx > goobj2.PkgIdxSelf || (lsym.PkgIdx == goobj2.PkgIdxInvalid && !lsym.Indexed()) || s.Linkname != "" {

What did you see instead?

Only returns references in the source file (objfile.go).

vscode-xref-repro-shellscript.txt

What exactly do you mean by an installation archive? Why not just tar up the contents of the CMAKE_INSTALL_PREFIX directory once the installation ninja target is complete?

Would you be able to post instructions and/or source code so that we can reproduce this?

The panic is happening while the linker is reading relocations out of an object file -- the offset it reads from the file is insane, which triggers the bounds range error. It would probably be a good idea to try to rule out the possibility of a corrupted object file (e.g. go clean -cache and rebuild).

I am unable to reproduce this on my work machine (Debian/amd64) using Go 1.15.2. Is there anything unusual about your configuration? The error in question seems very improbable; "eface" is a key type used in the Go runtime -- if the linker can't find a type descriptor for it, this suggests a damaged/corrupted installation (bad runtime.a).

For some reason it looks as though some of the cmake config checks that LLVM runs (this is outside of the gollvm cmake rules) are using -Werror (but only for C++ compiles, not for C compiles). So what's happening is that the cmake rules run "clang++ -c foo.cpp -fuse-ld=..." and clang warns because the linker option is not used ... and then the warning is turned into an error. I guess I would tack on "-Qunused-arguments" in this situation in addition to "-fuse-ld=xxx".

Also, not sure what the intended effect of "-fuse-ld=ld" is. The argument of this option should be one of {bfd,gold,ldd}.

Maybe related: https://superuser.com/questions/948808/why-is-moving-files-between-btrfs-subvolumes-an-expensive-operation

could you open a PR issue?

There are many features not supported by Gollvm; I am not sure what good it would do to create a laundry list of issues enumerating all the things that could be done but are not. Doing this simple clutters up the Go issue tracker as I see it (more productive I think to have these sorts of discussions (we should implement feature ABC instead of XYZ) on a mailing list like go-dev.

It also goes without saying that the main obstacle here is contributor bandwidth. I myself have a very limited amount of time to spend working on Gollvm-- there are many other higher-priority tasks for me at the moment. If we want to add new features, someone has to be willing to devote time to implementing them.

Especially surprising given the failure mode. Looking at

cmd/link/internal/loader.Reloc2.Sym(0x7f06bc66f241, 0xc000da2900, 0xc000073c00, 0x0, 0x4)
	/usr/local/go/src/cmd/link/internal/loader/loader.go:83 +0x22 fp=0xc000106d40 sp=0xc000106d10 pc=0x517262

that implies that Relocs.At2() returned a Reloc2 value with '*goobj2.Reloc' field set to a garbage value, which looking at the code seems impossible. Very strange.

As Ian mentioned, "go tool cover" is specific to the GC toolchain.

Correct, gollvm doesn't implement "-pg" at the moment. I would imagine it's doable, I think nobody has bothered to look into it yet.

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

<pre> $ go version go version go1.13.8 linux/amd64 </pre>

Does this issue reproduce with the latest release?

Yes. I am not able to use/import the external go packages with gccgo or llvm-goc. I want to generate LLVM IR for a go program that uses an external package. It errors out saying package not found.

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/home/virtualvms/.cache/go-build" GOENV="/home/virtualvms/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/var/hyperledger/gochaincode/gopath" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/lib/go-1.13" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64" GCCGO="/usr/bin/gccgo" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build376284318=/tmp/go-build -gno-record-gcc-switches" </pre></details>

What did you do?

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. --> I want to generate LLVM IR for a go program that uses an external package say a package/module like github.com/hyperledger/fabric-chaincode-go. I am using GO11MODULE to download and vendor the packages. It still fails saying the modules are not found. All required files are in the same folder.

payment_main.go:10:52: error: import file 'github.com/hyperledger/fabric-chaincode-go/pkg/cid' not found
payment_main.go:11:49: error: import file 'github.com/hyperledger/fabric-chaincode-go/shim' not found
payment_main.go:12:2: error: import file 'github.com/hyperledger/fabric-protos-go/peer' not found

What did you expect to see?

Proper build and LLVM IR of my go program

What did you see instead?

Error, External package not found during build process of gccgo or llvm-goc.

Closing this issue out, I don't see anything actionable here.

Most likely a shell problem? If you can try rerunning with SHELL=/bin/bash that might confirm.

Ping

Hi. I was building internals of linkerd and caught a bug, during the build phase, for a protobuf/grpc related package.

What version of Go are you using (go version)?

<pre> $ go version </pre>

go version go1.15rc2 gollvm LLVM 12.0.0git linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env

</pre>

GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/home/oceanfish81/.cache/go-build" GOENV="/home/oceanfish81/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/oceanfish81/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/home/oceanfish81/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/tools" GCCGO="/usr/local/bin/llvm-goc" AR="ar" CC="/usr/bin/clang" CXX="/usr/bin/clang++" CGO_ENABLED="1" GOMOD="/home/oceanfish81/linkerd2/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build000799681=/tmp/go-build -gno-record-gcc-switches -funwind-tables"

</details>

What did you do?

~/linkerd2/pkg/tap$ go build

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

What did you expect to see?

Required sources should be pulled and all packages should be compiled, using llvm-goc, as expected.

What did you see instead?

go: downloading github.com/sirupsen/logrus v1.6.0 go: downloading k8s.io/apimachinery v0.17.4 go: downloading k8s.io/client-go v0.17.4 go: downloading github.com/servicemeshinterface/smi-sdk-go v0.3.0 go: downloading k8s.io/kube-aggregator v0.17.4 go: downloading github.com/prometheus/client_golang v1.2.1 go: downloading k8s.io/api v0.17.4 go: downloading github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4 go: downloading github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 go: downloading k8s.io/apiextensions-apiserver v0.17.4 go: downloading github.com/prometheus/procfs v0.0.5 go: downloading github.com/prometheus/common v0.7.0 go: downloading github.com/Azure/go-autorest/autorest v0.9.0 go: downloading k8s.io/utils v0.0.0-20191114184206-e782cd3c129f go: downloading github.com/gogo/protobuf v1.3.0 go: downloading github.com/googleapis/gnostic v0.3.1 go: downloading github.com/Azure/go-autorest/tracing v0.5.0 go: downloading github.com/Azure/go-autorest/logger v0.1.0 go: downloading github.com/gophercloud/gophercloud v0.1.0 go: downloading github.com/docker/spdystream v0.0.0-20160310174837-449fdfce4d96 go: downloading github.com/Azure/go-autorest/autorest/adal v0.5.0 go: downloading k8s.io/kube-openapi v0.0.0-20191107075043-30be4d16710a go: downloading github.com/Azure/go-autorest/autorest/date v0.1.0 go: downloading github.com/evanphx/json-patch v4.5.0+incompatible go: downloading github.com/cespare/xxhash v1.1.0 go: downloading github.com/cespare/xxhash/v2 v2.1.0 go: downloading github.com/dgrijalva/jwt-go v3.2.0+incompatible go: downloading github.com/json-iterator/go v1.1.8 go: downloading github.com/pkg/errors v0.9.1

google.golang.org/grpc/internal/syscall

../../../go/pkg/mod/google.golang.org/[email protected]/internal/syscall/syscall_linux.go:50:28: error: reference to undefined identifier 'syscall.RUSAGE_SELF'

It seems that an error is related to func GetRusage() (rusage *Rusage) .

Ivan

This should be fixed now at head. Please reopen if not.

Hello again. I was trying to build an archived project, rkt - so I expected to use stable source code base, for testing purposes.

What version of Go are you using (go version)?

<pre> $ go version

</pre> go version go1.15rc2 gollvm LLVM 12.0.0git linux/amd64

Does this issue reproduce with the latest release?

Probably.

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> $ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/home/oceanfish81/.cache/go-build" GOENV="/home/oceanfish81/.config/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/oceanfish81/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/home/oceanfish81/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/tools" GCCGO="/usr/local/bin/llvm-goc" AR="ar" CC="/usr/bin/clang" CXX="/usr/bin/clang++" CGO_ENABLED="1" GOMOD="" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build520867872=/tmp/go-build -gno-record-gcc-switches -funwind-tables" </summary><br><pre> $ go env

</pre>

</details>

What did you do?

I was searching for Go files - and tried to build inside a sub-folder:

~/rkt/common$ go build

_/home/oceanfish81/rkt/common

./common.go:480:29: error: reference to undefined identifier 'syscall.DT_UNKNOWN'

@thanm , that's not the only constant for auto-generation (as for today).

./common.go:530:3: error: goarm is not a function; '//go:linkname' is only supported for functions

@cherrymui , are you familiar with the cause of this bug?

Ivan

I don't have anything to add above and beyond what I mentioned in https://www.mail-archive.com/[email protected]/msg31266.html

I'll update my patch in 254941 for DT_UNKNOWN.

Regarding go:linkname only supported for functions -- this is a known issue, see https://github.com/golang/go/issues/30771 for example. The rkt code in question (reaching into the runtime to grab references to unexported variables) is not something that we want to encourage-- a package like this can potentially break on any new Go release as runtime internals change.

OK, it looks as though -fsplit-stack is being added explicitly by the Go command (I thought this was something specific to gollvm, but I see the same thing with gccgo). For the moment I don't have a good workaround, other than using GCC as your C compiler instead of clang.

Thanks for posting the zip file with build artifacts. I can see the problem now; I'll send a patch shortly to address this.

This is a known limitation of the LLVM back end and its implementation of -fsplit-stack; to the best of my knowledge this restriction dates back to the original segmented stack support added in 2011.

One thing that I can't quite understand is how -fsplit-stack is winding up on the command line in the first place, however. In your "go env" output I see

GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build762480405=/tmp/go-build -gno-record-gcc-switches -funwind-tables"

which does not include "-fsplit-stack" -- when building C files for cgo, there is no reason to include this option.

I can't reproduce this -- works ok for me.

As Ian mentions, syscall.RUSAGE_SELF is auto-generated as part of the build. If you examine your gollvm build area, you should see it here:

$ cd <build_area_root>
$ fgrep RUSAGE_SELF tools/gollvm/libgo/sysinfo.go
const _RUSAGE_SELF = 0
const RUSAGE_SELF = _RUSAGE_SELF
$

If not, it sounds as though something went haywire with your build.

Does the directory $GOPATH/src/golang.org/x/net/.git exist?

Yes, $GOPATH/src/golang.org/x/net/.git exists.

What about $GOPATH/src/golang.org/x/net/http/httpguts/.git?

No, that does not exist.

/usr/local is coming from CMake itself, not from gollvm:

https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html

Regarding "lib" vs "lib64" -- lib64 is currently cooked into a number of places in the driver and cmake scripting:

thanm@cetina:/ssd2/llvm-project/llvm-project/llvm/tools/gollvm$ git grep lib64
README.md:% export LD_LIBRARY_PATH=/tmp/gollvm-install/lib64
cmake/modules/AddGollvm.cmake:#       under 'lib64' as well, further fix required
cmake/modules/AddGollvm.cmake:set(libsubdir "lib64")
cmake/modules/GoVars.cmake:# Driver::installedLibDir honors ./lib64 only
driver/Driver.cpp:    // NOTE: set aarch64's lib dir to "lib64" temporarily, until necessary
driver/Driver.cpp:      llvm::sys::path::append(ldir, "../lib64");
driver/Driver.cpp:      llvm::sys::path::append(ldir, "../lib64");
driver/Driver.h:  // Installed lib dir (binary dir above plus ../lib64)
driver/GccUtils.cpp:      s.libdirs.push_back("lib64");
driver/LinuxToolChain.cpp:  return triple.isArch32Bit() ? "lib" : "lib64";
driver/LinuxToolChain.cpp:      LibDir = "lib64";

This could certainly be reworked, but it would require some doing. Happy to review if you would like to send a patch.

I took a look at this -- I can reproduce, but I don't have any insight into what's going wrong. I don't see any replace directives at least as far as I can tell.

Most likely a shell problem? If you can try rerunning with SHELL=/bin/bash that might confirm.

What version of Go are you using (go version)?

<pre> $ go version 1.15 </pre>

Does this issue reproduce with the latest release?

I think so.

What operating system and processor architecture are you using (go env)?

Ubuntu 20 x86_64

What did you do?

Just an ordinary compilation of gollvm project, using LLVM 11.0.0-rc2 Passed libc++ into C++ compiler flags: -DCMAKE_CXX_FLAGS=-stdlib=libc++ . And limited myself to -DLLVM_TARGETS_TO_BUILD=X86 .

What did you expect to see?

Normal build process, using Ninja.

What did you see instead?

[1/1411] Building CXX object tools/gollvm/bridge/CMakeFiles/LLVMCppGoFrontEnd.dir//gofrontend/go/go.cc.o FAILED: tools/gollvm/bridge/CMakeFiles/LLVMCppGoFrontEnd.dir//gofrontend/go/go.cc.o /usr/bin/clang++ -DGTEST_HAS_RTTI=0 -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Itools/gollvm/bridge -I/media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm/llvm-project/llvm/tools/gollvm/bridge -Iinclude -I/media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm/llvm-project/llvm/include -Itools/gollvm/external/install/include -I/media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm/llvm-project/llvm/tools/gollvm/gofrontend/go -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -Os -DNDEBUG -fno-exceptions -fno-rtti -std=c++14 -MD -MT tools/gollvm/bridge/CMakeFiles/LLVMCppGoFrontEnd.dir//gofrontend/go/go.cc.o -MF tools/gollvm/bridge/CMakeFiles/LLVMCppGoFrontEnd.dir//gofrontend/go/go.cc.o.d -o tools/gollvm/bridge/CMakeFiles/LLVMCppGoFrontEnd.dir/__/gofrontend/go/go.cc.o -c /media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm/llvm-project/llvm/tools/gollvm/gofrontend/go/go.cc In file included from /media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm/llvm-project/llvm/tools/gollvm/gofrontend/go/go.cc:15: /media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm/llvm-project/llvm/tools/gollvm/gofrontend/go/gogo.h:3625:3: warning: 'type' overrides a member function but is not marked 'override' [-Wsuggest-override] type(Type*); ^ /media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm/llvm-project/llvm/tools/gollvm/gofrontend/go/gogo.h:3587:3: note: overridden virtual function is here type(Type*); ^ /media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm /llvm-project/llvm/tools/gollvm/gofrontend/go/go.cc:49:13: error: no member named 'need_eqtype' in 'go_create_gogo_args' if (args->need_eqtype) ~~~~ ^ /media/ubuntu/4d5fa4ce-dc9b-4cb0-934c-72533ffc1586/gollvm /llvm-project/llvm/tools/gollvm/gofrontend/go/go.cc:50:35: error: no member named 'need_eqtype' in 'go_create_gogo_args' ::gogo->set_need_eqtype(args->need_eqtype); ~~~~ ^ 1 warning and 2 errors generated.

This should be fixed now with CL's 251777 + 251778 (just submitted). Let me know if you see any other issues.

Thanks. I'll send a patch.

In the C/C++ compiler world, one thing that I've seen is that the presence or absence of "-g" can cause unexpected differences in the code generated by the compiler (e.g. "cc -g -O myfile.c" produces different assembly from "cc -O myfile.c"). This has come up many times in nearly every compiler I've worked on.

Once we have "-debug={true,false}" up and running for the Go compiler, I think it would be useful to enhance our tools testing to:

• add some sort of compatibility comparable to "toolstash -check" that verifies that code generated with and without "-debug=..." is the same (to be used by compiler developers in pre-submit testing)

• add some set of builders that enforce the same invariant

My 2 cents.

Possibly DWARF related? What happens when "-w" is added to linker options?

Here is some more detail on what's happening in the compiler with this bug. There are two problems, one easy to fix and the other more difficult.

The first problem is that the code that computes ranges for inlined routines is not doing the book-keeping properly to insure that child ranges are included in parent ranges when handling nested inlines. This problem triggers the verifier's "DIE address ranges are not contained in its parent's ranges" error. I sent CL 248724 to fix this.

The second problem is trickier. The code that tracks variable scopes in the compiler is more or less independent from the code that handles inlining, meaning that when you have an inlined call whose callsite is positioned within a given lexical scope, the inlined routine DIE doesn't wind up as a child of the scope. This triggers a different error ("DIEs have overlapping address ranges").

Here is a reduced testcase that illustrates the second problem:

File a.go:

package a

func S(f, l, o int, v int) (err error) {
	var n = int32(v)
	return s(f, l, o, &n, 4)
}

func s(f, l, o int, v *int32, sz int) (err error) {
	defer func() { *v = 3 }()
	return nil
}

begin b.go:

package main

import "issue33188/a"

var gf, gl, gof int

func main() {
	for {
		var b [16]int
		e := a.S(gf, gl, gof, 33)
		if e == nil {
			continue
		}
		gf += 9
		b[gf&3] += 1
	}
}

Here's an abstracted version of the DWARF for main.main you get in this situation:

<1><f54>: Abbrev Number: 3 (DW_TAG_subprogram)
   <f55>   DW_AT_name        : main.main
   <f5f>   DW_AT_low_pc      : 0x45cd80
   <f67>   DW_AT_high_pc     : 0x45ce50
   <f6f>   DW_AT_frame_base  : 1 byte block: 9c 	(DW_OP_call_frame_cfa)
   <f71>   DW_AT_decl_file   : 0x2
   <f75>   DW_AT_external    : 1
<2><f76>: Abbrev Number: 20 (DW_TAG_lexical_block)
   <f77>   DW_AT_ranges      : 0x40
<3><f7b>: Abbrev Number: 11 (DW_TAG_variable)
   <f7c>   DW_AT_name        : b
   <f7e>   DW_AT_decl_line   : 9
   <f7f>   DW_AT_type        : <0x3cdc3>
   <f83>   DW_AT_location    : 0x0 (location list)
<3><f87>: Abbrev Number: 0
<2><f88>: Abbrev Number: 6 (DW_TAG_inlined_subroutine)
   <f89>   DW_AT_abstract_origin: <0xf0f>
   <f8d>   DW_AT_low_pc      : 0x45cde8
   <f95>   DW_AT_high_pc     : 0x45ce24
   <f9d>   DW_AT_call_file   : 0x2
   <fa1>   DW_AT_call_line   : 10
<3><fa2>: Abbrev Number: 19 (DW_TAG_formal_parameter)
   <fa3>   DW_AT_abstract_origin: <0xf21>
   <fa7>   DW_AT_location    : 0x35 (location list)
<3><fab>: Abbrev Number: 19 (DW_TAG_formal_parameter)
   <fac>   DW_AT_abstract_origin: <0xf29>
   <fb0>   DW_AT_location    : 0x68 (location list)
<3><fb4>: Abbrev Number: 19 (DW_TAG_formal_parameter)
   <fb5>   DW_AT_abstract_origin: <0xf31>
   <fb9>   DW_AT_location    : 0x9b (location list)
<3><fbd>: Abbrev Number: 14 (DW_TAG_variable)
   <fbe>   DW_AT_abstract_origin: <0xf4b>
   <fc2>   DW_AT_location    : 0xce (location list)
<3><fc6>: Abbrev Number: 0
<2><fc7>: Abbrev Number: 0

where the ranges for the lexical block containing "b" look like

             [0x000000000045cdaf, 0x000000000045cdf0)
             [0x000000000045ce24, 0x000000000045ce46))

which overlaps with the inlined routine but doesn't entirely contain it (and in addition the two DIEs in question are siblings within the main.main DIE, as opposed to having the scope parent the inline.

Fixing this is a good deal more complicated, since

Closing this bug, since the dev.link branch is now merged into master.

What version of Go are you using (go version)?

<pre> $ go version go version devel +60f7876502 Sat Jun 20 08:40:13 2020 +0000 linux/amd64 </pre>

Does this issue reproduce with the latest release?

yes.

What operating system and processor architecture are you using (go env)?

linux/amd64 but this problem applies on other archs as well

What did you do?

Build a small program, e.g. "go build himom.go"

package main

import (
	"fmt"
)

func main() {
	fmt.Printf("hi mom\n")
}

Then examine generated DWARF with "objdump -dl":

$ objdump -t himom | fgrep main.main
000000000049a0e0 g     F .text	0000000000000070 main.main
$ objdump --wide -dl --start-address=0x497f00 --stop-address=0x497f70 himom

himom:     file format elf64-x86-64


Disassembly of section .text:

0000000000497f00 <main.main>:
main.main():
/usr/local/google/home/thanm/himom.go:7
  497f00:	64 48 8b 0c 25 f8 ff ff ff 	mov    %fs:0xfffffffffffffff8,%rcx
  497f09:	48 3b 61 10          	cmp    0x10(%rcx),%rsp
  497f0d:	76 5a                	jbe    497f69 <main.main+0x69>
  497f0f:	48 83 ec 58          	sub    $0x58,%rsp
  497f13:	48 89 6c 24 50       	mov    %rbp,0x50(%rsp)
  497f18:	48 8d 6c 24 50       	lea    0x50(%rsp),%rbp
/usr/local/google/home/thanm/himom.go:8
  497f1d:	48 8b 05 6c 6a 0b 00 	mov    0xb6a6c(%rip),%rax        # 54e990 <os.Stdout>
/ssd2/ygo/src/fmt/print.go:213
  497f24:	48 8d 0d 15 1b 04 00 	lea    0x41b15(%rip),%rcx        # 4d9a40 <go.itab.*os.File,io.Writer>
fmt.Printf():
/ssd2/ygo/src/fmt/print.go:213
  497f2b:	48 89 0c 24          	mov    %rcx,(%rsp)
  497f2f:	48 89 44 24 08       	mov    %rax,0x8(%rsp)
  497f34:	48 8d 05 d6 30 02 00 	lea    0x230d6(%rip),%rax        # 4bb011 <go.string.*+0x641>
  497f3b:	48 89 44 24 10       	mov    %rax,0x10(%rsp)
  497f40:	48 c7 44 24 18 07 00 00 00 	movq   $0x7,0x18(%rsp)
  497f49:	48 c7 44 24 20 00 00 00 00 	movq   $0x0,0x20(%rsp)
  497f52:	0f 57 c0             	xorps  %xmm0,%xmm0
  497f55:	0f 11 44 24 28       	movups %xmm0,0x28(%rsp)
  497f5a:	e8 e1 82 ff ff       	callq  490240 <fmt.Fprintf>
main.main():
/usr/local/google/home/thanm/himom.go:8
  497f5f:	48 8b 6c 24 50       	mov    0x50(%rsp),%rbp
  497f64:	48 83 c4 58          	add    $0x58,%rsp
  497f68:	c3                   	retq   
./<autogenerated>:1
  497f69:	e8 52 90 fc ff       	callq  460fc0 <runtime.morestack_noctxt>
  497f6e:	eb 90                	jmp    497f00 <main.main>

What did you expect to see?

Call to "runtime.morestack" has a source position (file/line) the same as the prolog of the function.

What did you see instead?

Source position is "autogenerated:1":

./<autogenerated>:1
  497f69:	e8 52 90 fc ff       	callq  460fc0 <runtime.morestack_noctxt>
  497f6e:	eb 90                	jmp    497f00 <main.main>

This can interfere with debugging and profiling.

One could argue that since this is compiler-generated code it's ok to give it an autogenerated source position (similar to things like method wrappers), but there is also a strong argument to be made that the morestack call is logically part of the function prolog (even though it is positioned in the cold code at the tail end of the function).

It also turns out that we haven't always marked this code with "autogenerated" -- in previous versions of Go it was being given the correct position. Here is the tail end of the assembly when built with 1.12:

/usr/local/google/home/thanm/himom.go:7
  488a9a:	callq  44f300 <runtime.morestack_noctxt>
  488a9f:	jmp    488a30 <main.main>

It looks as though the problem was introduced when the main part of DWARF line table generation was moved from the linker in to the compiler. This code here:

https://go.googlesource.com/go/+/d1a186d29ce9d917dda7c66cfaee7788f88e7b9e/src/cmd/internal/obj/dwarf.go#112

generates a sequence at the end of each function to reset the state of the line table back to its default settings. This code doesn't advance the PC before applying the new file and line, however, meaning that it overwrites the file/line for the last row in the line table, which is what's causing the "autogenerated:1" to be picked up.

This could be fixed in a couple of ways: one would be to emit an end of sequence op after each function (in which case there would be no manual file/line settting needed). A second option would be to add code to manually advanced the PC past the end of the last instruction before emitting the set file/set line ops.

Hmm, I think I may need to generate a new CL instead of hijacking the old one. Stay tuned.

Thanks Dmitri. This is being actively worked on. I am waiting on a couple more CLs from Jeremy before finalizing the merge.

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

<pre> $ go version go version go1.15rc1 gollvm LLVM 12.0.0git linux/amd64 </pre>

Does this issue reproduce with the latest release?

Yes, I run the configure step of gollvm in a docker container (ubuntu 18.04), but fail with following logs:

-- Performing Test SPLIT_STACK_FUNCTIONAL
-- Performing Test SPLIT_STACK_FUNCTIONAL - Success
-- starting libgo configuration.
CMake Error at tools/gollvm/cmake/modules/GoVars.cmake:41 (message):
  fatal: shell missing or not functional
Call Stack (most recent call first):
  tools/gollvm/libgo/CMakeLists.txt:11 (include)


-- Configuring incomplete, errors occurred!
See also "/root/release/CMakeFiles/CMakeOutput.log".
See also "/root/release/CMakeFiles/CMakeError.log".

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env

</pre></details>

What did you do?

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

I fetch a fresh gollvm to a docker container (ubuntu 18.04) then configure the building by

cmake ../llvm -DCMAKE_INSTALL_PREFIX=/root/gollvm -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 -DLLVM_ENABLE_ASSERTIONS=On -DLLVM_ENABLE_RTTI=On -DLLVM_USE_LINKER=gold -G Ninja

What did you expect to see?

no errors

What did you see instead?

configuration failed

Lots of comments and observations here, sounds like the next step is to write some patches.

Yes please, feel free to submit a patch. I seem to recall at least one other person running into this same issue. Another option would be to require folks to set SHELL prior to running cmake (e.g. update the build instructions).

Does this issue reproduce with the latest release?

Yes (with latest commit to master).

What operating system and processor architecture are you using (go env)?

Ubuntu Focal Fossa, amd64

What did you do?

I followed the instructions on the gollvm landing page.

What did you expect to see?

The project builds cleanly.

What did you see instead?

I got this output

-- Could NOT find LibXml2 (missing: LIBXML2_LIBRARY LIBXML2_INCLUDE_DIR) 
-- Native target architecture is X86
-- Threads enabled.
-- Doxygen disabled.
-- Go bindings disabled.
-- Ninja version: 1.10.0
-- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) 
-- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) 
-- OCaml bindings disabled.
-- LLVM host triple: x86_64-unknown-linux-gnu
-- LLVM default target triple: x86_64-unknown-linux-gnu
-- Building with -fPIC
-- Constructing LLVMBuild project information
-- Targeting AArch64
-- Targeting AMDGPU
-- Targeting ARM
-- Targeting AVR
-- Targeting BPF
-- Targeting Hexagon
-- Targeting Lanai
-- Targeting Mips
-- Targeting MSP430
-- Targeting NVPTX
-- Targeting PowerPC
-- Targeting RISCV
-- Targeting Sparc
-- Targeting SystemZ
-- Targeting WebAssembly
-- Targeting X86
-- Targeting XCore
CMake Error at tools/gollvm/cmake/modules/AddGollvm.cmake:37 (message):
  C compiler does not support -fsplit-stack
Call Stack (most recent call first):
  tools/gollvm/CMakeLists.txt:21 (include)


-- starting libgo configuration.
CMake Error at tools/gollvm/cmake/modules/LibbacktraceUtils.cmake:19 (message):
  Support for mmap() is required -- setup failed.
Call Stack (most recent call first):
  tools/gollvm/libgo/CMakeLists.txt:45 (setup_libbacktrace)


CMake Error at tools/gollvm/cmake/modules/LibffiUtils.cmake:27 (message):
  Support for mmap() is required -- setup failed.
Call Stack (most recent call first):
  tools/gollvm/libgo/CMakeLists.txt:49 (setup_libffi)


-- Libgo: creating stdlib package targets
-- Libgo: generating check targets
-- libgo configuration complete.
-- starting gotools configuration.
-- gotools: generating check targets
-- gotools configuration complete.
-- Registering Bye as a pass plugin (static build: OFF)
-- Failed to find LLVM FileCheck
-- Version: 0.0.0
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Configuring incomplete, errors occurred!
See also "/home/gopher/workarea/build-debug/CMakeFiles/CMakeOutput.log".
See also "/home/gopher/workarea/build-debug/CMakeFiles/CMakeError.log".
ninja: error: loading 'build.ninja': No such file or directory

The output of CMakeError.log is around 2.5k lines. Here is a snippet:

Determining if the include file malloc/malloc.h exists failed with the following output:
Change Dir: /home/gopher/workarea/build-debug/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/ninja cmTC_4ce15 && [1/2] Building C object CMakeFiles/cmTC_4ce15.dir/CheckIncludeFile.c.o
FAILED: CMakeFiles/cmTC_4ce15.dir/CheckIncludeFile.c.o 
/usr/bin/cc    -o CMakeFiles/cmTC_4ce15.dir/CheckIncludeFile.c.o   -c CheckIncludeFile.c
CheckIncludeFile.c:1:10: fatal error: malloc/malloc.h: No such file or directory
    1 | #include <malloc/malloc.h>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

Determining if the include file valgrind/valgrind.h exists failed with the following output:
Change Dir: /home/gopher/workarea/build-debug/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/ninja cmTC_d0b71 && [1/2] Building C object CMakeFiles/cmTC_d0b71.dir/CheckIncludeFile.c.o
FAILED: CMakeFiles/cmTC_d0b71.dir/CheckIncludeFile.c.o 
/usr/bin/cc    -o CMakeFiles/cmTC_d0b71.dir/CheckIncludeFile.c.o   -c CheckIncludeFile.c
CheckIncludeFile.c:1:10: fatal error: valgrind/valgrind.h: No such file or directory
    1 | #include <valgrind/valgrind.h>
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.

Has anyone managed to build gollvm? Or have a Docker image? The Docker images publicly available are somewhat old and also gave me bugs.

I've sent in a couple of CLs that should address the build issues, and @erifan also sent a fix. Things should be buildable using your Docker container now. Note that you'll need to include m4 in the dependences, and insure that SHELL env is set. Let me know if you have additional problems.

If you can post detailed instructions for creating the container (etc) I can give that a try; I do have access to a macbook.

Well, I am stumped at the moment. The log files you posted are functionally identical to the ones from one of my recent builds, but your run seems to fail and mine does not. About all I can suggest at this point is cleaning our your build area (rm -r CMakeFiles CMakeCache.txt) and starting from a clean slate, since I can't find anything actionable in your logs at all.

Thanks. Would it be possible to share the last couple of pages of the CMakeFiles/CMakeOutput.log file? Maybe that might given an indication as to why the cmake run is failing (since I don't see any red flags in the transcript you posted).

You wrote:

Commit: de65b356dc2 (HEAD -> master, origin/master, origin/HEAD) [VectorCombine] add/use pass-level IRBuilder

That's the LLVM commit hash. I'm interested in the Gollvm hash, e.g.

cd <llvm_project_root>t/llvm/tools/gollvm
git log -1 --oneline

It should be at 2ded6bd. Thanks.

I am still unable to reproduce this latest failure. When I test at tip using a compiler that hard-wires on -fcf-protection=branch I get a clean build.

For anyone still encountering this problem, please rerun your cmake command and look for the following in the trace output:

-- default linker set to "gold"
-- trying -fcf-protection=none workaround
-- applying -fcf-protection=none workaround
-- starting libgo configuration.

If you don't see this, please post what you are seeing from cmake. It would also help to know which specific commit you're picking up in the gollvm repo (e.g. "git log -1 --oneline"). Thanks.

Thanks for sending this bug report.

It would be helpful to know how your ZFS storage pool is set up. Could you please share the output of "zfs get all" or "zfs get recordsize"?