profile
viewpoint
nodejs Spring Boot React Rust tensorflow

Ask questionsruntime: crash with "invalid pc-encoded table"

What version of Go are you using (go version)?

go 1.11

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/users/kai.hayashi/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build387775402=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

I am running a fairly run-of-the-mill production api server. I have observed one crash so far on this code since upgrading to go1.11 with fatal error: invalid runtime symbol table. I am running a library in this server code that periodically captures a cpu, block, and mutex profile every few minutes. The sample rates on the mutex profile is 1/10 and 1 block profile every second.

What did you expect to see?

No crashes

What did you see instead?

runtime: invalid pc-encoded table [...]
fatal error: invalid runtime symbol table
goroutine 0 [idle]:
runtime.throw(0x10501de, 0x1c)
      /usr/local/go/src/runtime/panic.go:608 +0x72 fp=0xc00064f258 sp=0xc00064f228 pc=0x42c2b2
runtime.pcvalue(0x179ed90, 0x19f6800, 0xXXXXXXXX005ea0e7, 0xaaf94c, 0xc00064f480, 0xXXXXXXXXXXXXXX01, 0xXXXXXXXX00000000)
      /usr/local/go/src/runtime/symtab.go:791 +0x50a fp=0xc00064f300 sp=0xc00064f258 pc=0x449d3a
runtime.funcspdelta(0x179ed90, 0x19f6800, 0xaaf94c, 0xc00064f480, 0xXXXXXXXX00000000)
     /usr/local/go/src/runtime/symtab.go:843 +0x5f fp=0xc00064f370 sp=0xc00064f300 pc=0x44a1ef
runtime.gentraceback(0x45d5d5, 0xc0005f3e00, 0x0, 0xc0004d1080, 0x0, 0xc00064f6d8, 0x40, 0x0, 0x0, 0x6, ...)
      /usr/local/go/src/runtime/traceback.go:208 +0x1828 fp=0xc00064f678 sp=0xc00064f370 pc=0x450ca8
runtime.sigprof(0x45d5d5, 0xc0005f3e00, 0x0, 0xc0004d1080, 0xc000244380)
      /usr/local/go/src/runtime/proc.go:3823 +0x4b6 fp=0xc00064f928 sp=0xc00064f678 pc=0x436cd6
runtime.sighandler(0xc00000001b, 0xc00064fbf0, 0xc00064fac0, 0xc0004d1080)
      /usr/local/go/src/runtime/signal_sighandler.go:38 +0x73c fp=0xc00064f9b8 sp=0xc00064f928 pc=0x440dec

This crash is reminiscent of a bug I was hoping would be fixed with the 1.11 release related to https://github.com/golang/go/issues/24925. The crashes do seem less frequent than when I was running the same code under 1.10.

CC @heschik and @rhysh

golang/go

Answer questions rkuska

FYI; Still present with 1.12.0, same story as others, service with profiling (cloud.google.com/go/profiler)

Some tracebacks from the failure if it helps:

runtime.throw(0xee6853, 0x1c)
	/usr/local/go/src/runtime/panic.go:617 +0x72
runtime.pcvalue(0x13e4228, 0x1919980, 0x33984f, 0x85abc6, 0xc000c2b4d0, 0x1123701, 0x0)
	/usr/local/go/src/runtime/symtab.go:704 +0x53f
runtime.funcspdelta(0x13e4228, 0x1919980, 0x85abc6, 0xc000c2b4d0, 0xc000000000)
	/usr/local/go/src/runtime/symtab.go:756 +0x5f
runtime.gentraceback(0x45ddb6, 0x7f8ff17f9b90, 0x0, 0xc000483500, 0x0, 0xc000c2b728, 0x40, 0x0, 0x0, 0x6, ...)
	/usr/local/go/src/runtime/traceback.go:222 +0x1811
runtime.sigprof(0x45ddb6, 0x7f8ff17f9b90, 0x0, 0xc000483500, 0xc000ca4000)
	/usr/local/go/src/runtime/proc.go:3754 +0x425
runtime.sighandler(0xc00000001b, 0xc000c2bbf0, 0xc000c2bac0, 0xc000483500)
	/usr/local/go/src/runtime/signal_sighandler.go:38 +0x6e7
runtime.sigtrampgo(0x1b, 0xc000c2bbf0, 0xc000c2bac0)
	/usr/local/go/src/runtime/signal_unix.go:351 +0x1f2
runtime: unexpected return pc for runtime.sigtramp called from 0xc5a5e0

runtime.sigtramp(0x7, 0x0, 0xc000c24000, 0xc000000000, 0x8000, 0x136a7c658dead1, 0x7fff2a5b6080, 0x1, 0xbc9fb4be, 0x2, ...)
	/usr/local/go/src/runtime/sys_linux_amd64.s:357 +0x43

goroutine 1177223 [GC worker (idle)]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:311 fp=0xc005591760 sp=0xc005591758 pc=0x45a260
runtime.gcBgMarkWorker(0xc000054000)
	/usr/local/go/src/runtime/mgc.go:1890 +0x1be fp=0xc0055917d8 sp=0xc005591760 pc=0x41d69e
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc0055917e0 sp=0xc0055917d8 pc=0x45c331
created by runtime.gcBgMarkStartWorkers
	/usr/local/go/src/runtime/mgc.go:1784 +0x77

goroutine 1 [chan receive, 1884 minutes]:
main.main()
	/conveyance-store/cmd/storage/conveyance-api.go:274 +0x1173

goroutine 19 [syscall, 1884 minutes]:
os/signal.signal_recv(0x0)
	/usr/local/go/src/runtime/sigqueue.go:139 +0x9c
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
	/usr/local/go/src/os/signal/signal_unix.go:29 +0x41

goroutine 367539921 [IO wait, 110 minutes]:
internal/poll.runtime_pollWait(0x7f906813c858, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc00ae38a98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc00ae38a80, 0xc00d4bb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc00ae38a80, 0xc00d4bb000, 0x1000, 0x1000, 0xc000048500, 0xc0089ebb08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000844440, 0xc00d4bb000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc00f5884b0, 0xc00d4bb000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc00e496960)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc00e496960, 0x4, 0x0, 0x0, 0x0, 0x0, 0x4bceba)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc00a088820, 0x104e480, 0xc00bc74f80, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc00a088820, 0x104e480, 0xc00bc74f80)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 378316985 [IO wait, 51 minutes]:
internal/poll.runtime_pollWait(0x7f9058038d18, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc005612998, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc005612980, 0xc004a86000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc005612980, 0xc004a86000, 0x1000, 0x1000, 0xc00005af00, 0xc012171b08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000a3e2b8, 0xc004a86000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc0065b0ba0, 0xc004a86000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc004e38000)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc004e38000, 0x4, 0x0, 0x0, 0x0, 0x0, 0xfa)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc00a6c68c0, 0x104e480, 0xc00b83cd00, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc00a6c68c0, 0x104e480, 0xc00b83cd00)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 370175441 [IO wait, 97 minutes]:
internal/poll.runtime_pollWait(0x7f903c47d2b8, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc00778b298, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc00778b280, 0xc010efe000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc00778b280, 0xc010efe000, 0x1000, 0x1000, 0xc000064500, 0xc014bbbb08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000d0c440, 0xc010efe000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc00e1fdd10, 0xc010efe000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc003cca540)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc003cca540, 0x4, 0x0, 0x0, 0x0, 0x0, 0x4bceba)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc00c221cc0, 0x104e480, 0xc00555c300, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc00c221cc0, 0x104e480, 0xc00555c300)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 387659619 [IO wait]:
internal/poll.runtime_pollWait(0x7f9068245180, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc011601b18, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc011601b00, 0xc01339f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc011601b00, 0xc01339f000, 0x1000, 0x1000, 0xc000066a00, 0xc013789b08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc019cd4838, 0xc01339f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc00716e6f0, 0xc01339f000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc0045981e0)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc0045981e0, 0x4, 0x0, 0x0, 0x0, 0x0, 0x4bceba)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc0040c43c0, 0x104e480, 0xc00829ce80, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc0040c43c0, 0x104e480, 0xc00829ce80)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 375849207 [IO wait, 65 minutes]:
internal/poll.runtime_pollWait(0x7f9068413c30, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc00dc83918, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc00dc83900, 0xc01371c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc00dc83900, 0xc01371c000, 0x1000, 0x1000, 0xc000056500, 0xc006a9bb08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0007d4250, 0xc01371c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc00c9c4b70, 0xc01371c000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc011a97c80)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc011a97c80, 0x4, 0x0, 0x0, 0x0, 0x0, 0x4bceba)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc00566a0a0, 0x104e480, 0xc00103ee80, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc00566a0a0, 0x104e480, 0xc00103ee80)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 377139601 [IO wait, 58 minutes]:
internal/poll.runtime_pollWait(0x7f90682031d8, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc01184c298, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc01184c280, 0xc00d694000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc01184c280, 0xc00d694000, 0x1000, 0x1000, 0xc000080500, 0xc012221b08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0000106e8, 0xc00d694000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc001a828a0, 0xc00d694000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc0043db380)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc0043db380, 0x4, 0x0, 0x0, 0x0, 0x0, 0x4bceba)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc00b25f540, 0x104e480, 0xc007b37d00, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc00b25f540, 0x104e480, 0xc007b37d00)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 375323721 [IO wait, 68 minutes]:
internal/poll.runtime_pollWait(0x7f9058039058, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc008c1c018, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc008c1c000, 0xc01d0a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc008c1c000, 0xc01d0a2000, 0x1000, 0x1000, 0xc000064500, 0xc00a22bb08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0000101e0, 0xc01d0a2000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc00632c090, 0xc01d0a2000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc01c9ba000)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc01c9ba000, 0x4, 0x0, 0x0, 0x0, 0x0, 0x4bceba)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc007a61040, 0x104e480, 0xc0155d4240, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc007a61040, 0x104e480, 0xc0155d4240)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 371375030 [IO wait, 91 minutes]:
internal/poll.runtime_pollWait(0x7f9068063400, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc01485de98, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc01485de80, 0xc017f21000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc01485de80, 0xc017f21000, 0x1000, 0x1000, 0xc000058a00, 0xc0145e9b08, 0x6b05ca)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0008ce370, 0xc017f21000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
net/http.(*connReader).Read(0xc001a83f20, 0xc017f21000, 0x1000, 0x1000, 0x0, 0x0, 0x72)
	/usr/local/go/src/net/http/server.go:787 +0x107
bufio.(*Reader).fill(0xc01486a120)
	/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc01486a120, 0x4, 0x0, 0x0, 0x0, 0x0, 0x4bceba)
	/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*conn).readRequest(0xc002328820, 0x104e480, 0xc0053474c0, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/http/server.go:964 +0xb03
net/http.(*conn).serve(0xc002328820, 0x104e480, 0xc0053474c0)
	/usr/local/go/src/net/http/server.go:1819 +0x6a8
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2884 +0x2f4

goroutine 14 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000509000)
	/go/pkg/mod/[email protected]/stats/view/worker.go:146 +0xdd
created by go.opencensus.io/stats/view.init.0
	/go/pkg/mod/[email protected]/stats/view/worker.go:29 +0x57

goroutine 79 [IO wait]:
internal/poll.runtime_pollWait(0x7f9070054578, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc000476098, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000476080, 0xc000856000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc000476080, 0xc000856000, 0x1000, 0x1000, 0x43f5bd, 0xc0003f94f8, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000010110, 0xc000856000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
bufio.(*Reader).Read(0xc000403140, 0xc000103c68, 0x1, 0x9, 0x2, 0x3, 0x80)
	/usr/local/go/src/bufio/bufio.go:223 +0x22f
io.ReadAtLeast(0x1033de0, 0xc000403140, 0xc000103c68, 0x1, 0x9, 0x1, 0x38, 0xc00942d740, 0x0)
	/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/gocql/gocql.readHeader(0x1033de0, 0xc000403140, 0xc000103c68, 0x9, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/scylladb/[email protected]/frame.go:446 +0xab
github.com/gocql/gocql.(*Conn).recv(0xc000103c20, 0x0, 0x0)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:526 +0xf8
github.com/gocql/gocql.(*Conn).serve(0xc000103c20)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:490 +0x31
created by github.com/gocql/gocql.(*Session).dial
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:259 +0x749

goroutine 86 [select, 49 minutes]:
google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000509a80)
	/go/pkg/mod/google.golang.org/[email protected]/balancer_conn_wrappers.go:122 +0x110
created by google.golang.org/grpc.newCCBalancerWrapper
	/go/pkg/mod/google.golang.org/[email protected]/balancer_conn_wrappers.go:113 +0x14f

goroutine 4602 [IO wait, 1884 minutes]:
internal/poll.runtime_pollWait(0x7f90682db498, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc000685998, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000685980, 0xc001b0a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc000685980, 0xc001b0a000, 0x1000, 0x1000, 0x43f5bd, 0xc0009b40b8, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0008440c8, 0xc001b0a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
bufio.(*Reader).Read(0xc000b89080, 0xc000dddb48, 0x1, 0x9, 0x2, 0x3, 0x80)
	/usr/local/go/src/bufio/bufio.go:223 +0x22f
io.ReadAtLeast(0x1033de0, 0xc000b89080, 0xc000dddb48, 0x1, 0x9, 0x1, 0x38, 0xc000a0a2c0, 0x0)
	/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/gocql/gocql.readHeader(0x1033de0, 0xc000b89080, 0xc000dddb48, 0x9, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/scylladb/[email protected]/frame.go:446 +0xab
github.com/gocql/gocql.(*Conn).recv(0xc000dddb00, 0x0, 0x0)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:526 +0xf8
github.com/gocql/gocql.(*Conn).serve(0xc000dddb00)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:490 +0x31
created by github.com/gocql/gocql.(*Session).dial
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:259 +0x749

goroutine 88 [select]:
gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer.(*tracer).worker(0xc0000f5860)
	/go/pkg/mod/gopkg.in/!data!dog/[email protected]/ddtrace/tracer/tracer.go:149 +0x1a8
created by gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer.newTracer
	/go/pkg/mod/gopkg.in/!data!dog/[email protected]/ddtrace/tracer/tracer.go:136 +0x30f

goroutine 4987 [IO wait, 1884 minutes]:
internal/poll.runtime_pollWait(0x7f907009d010, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc000685898, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000685880, 0xc001aac000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc000685880, 0xc001aac000, 0x1000, 0x1000, 0x43f5bd, 0xc00024baf8, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000ce00c0, 0xc001aac000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
bufio.(*Reader).Read(0xc00173c840, 0xc001a12168, 0x1, 0x9, 0x2, 0x3, 0x80)
	/usr/local/go/src/bufio/bufio.go:223 +0x22f
io.ReadAtLeast(0x1033de0, 0xc00173c840, 0xc001a12168, 0x1, 0x9, 0x1, 0x38, 0xc000b3c740, 0x0)
	/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/gocql/gocql.readHeader(0x1033de0, 0xc00173c840, 0xc001a12168, 0x9, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/scylladb/[email protected]/frame.go:446 +0xab
github.com/gocql/gocql.(*Conn).recv(0xc001a12120, 0x0, 0x0)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:526 +0xf8
github.com/gocql/gocql.(*Conn).serve(0xc001a12120)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:490 +0x31
created by github.com/gocql/gocql.(*Session).dial
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:259 +0x749

goroutine 4447 [IO wait, 1884 minutes]:
internal/poll.runtime_pollWait(0x7f90700fbfe8, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc000685798, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000685780, 0xc001a0e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc000685780, 0xc001a0e000, 0x1000, 0x1000, 0x43f5bd, 0xc0000d0b38, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc0007d4158, 0xc001a0e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
bufio.(*Reader).Read(0xc001118ae0, 0xc001918288, 0x1, 0x9, 0x7f906855b6d8, 0x41, 0x80)
	/usr/local/go/src/bufio/bufio.go:223 +0x22f
io.ReadAtLeast(0x1033de0, 0xc001118ae0, 0xc001918288, 0x1, 0x9, 0x1, 0x38, 0xc000b1b040, 0x0)
	/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/gocql/gocql.readHeader(0x1033de0, 0xc001118ae0, 0xc001918288, 0x9, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/scylladb/[email protected]/frame.go:446 +0xab
github.com/gocql/gocql.(*Conn).recv(0xc001918240, 0x0, 0x0)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:526 +0xf8
github.com/gocql/gocql.(*Conn).serve(0xc001918240)
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:490 +0x31
created by github.com/gocql/gocql.(*Session).dial
	/go/pkg/mod/github.com/scylladb/[email protected]/conn.go:259 +0x749

goroutine 87 [select]:
github.com/googleapis/gax-go.Sleep(0x104e540, 0xc00024d200, 0x2540be400, 0x0, 0x0)
	/go/pkg/mod/github.com/googleapis/[email protected]+incompatible/invoke.go:55 +0xd2
cloud.google.com/go/profiler.(*agent).profileAndUpload(0xc000509ac0, 0x104e540, 0xc00024d200, 0xc00945a1c0)
	/go/pkg/mod/cloud.google.com/[email protected]/profiler/profiler.go:311 +0x5b5
cloud.google.com/go/profiler.pollProfilerService(0x104e540, 0xc00024d200, 0xc000509ac0)
	/go/pkg/mod/cloud.google.com/[email protected]/profiler/profiler.go:527 +0x6f
created by cloud.google.com/go/profiler.start
	/go/pkg/mod/cloud.google.com/[email protected]/profiler/profiler.go:204 +0x491

goroutine 30 [chan receive, 49 minutes]:
google.golang.org/grpc.(*addrConn).resetTransport(0xc0002d0000)
	/go/pkg/mod/google.golang.org/[email protected]/clientconn.go:1070 +0x9e5
created by google.golang.org/grpc.(*addrConn).connect
	/go/pkg/mod/google.golang.org/[email protected]/clientconn.go:684 +0xb6

goroutine 1818 [IO wait]:
internal/poll.runtime_pollWait(0x7f9068499008, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc000150718, 0x72, 0x1000, 0x1000, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc000150700, 0xc001068000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc000150700, 0xc001068000, 0x1000, 0x1000, 0x43f5bd, 0xc000168118, 0x0)
	/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000010020, 0xc001068000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:177 +0x69
bufio.(*Reader).Read(0xc000946540, 0xc000ddc288, 0x1, 0x9, 0x2, 0x3, 0x80)
	/usr/local/go/src/bufio/bufio.go:223 +0x22f
io.ReadAtLeast(0x1033de0, 0xc000946540, 0xc000ddc288, 0x1, 0x9, 0x1, 0x38, 0xc0157a5440, 0x0)
	/usr/local/go/src/io/io.go:310 +0x88
io.ReadFull(...)
	/usr/local/go/src/io/io.go:329
github.com/gocql/gocql.readHeader(0x1033de0, 0xc000946540, 0xc000ddc288, 0x9, 0x9, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/pkg/mod/github.com/scylladb/[email protected]/frame.go:446 +0xab
github.com/gocql/gocql.(*Conn).recv(0xc000ddc240, 0x0, 0x0)
useful!

Related questions

net/http: TestRetryRequestsOnError is flaky hot 242
net/http: TestTransportServerClosingUnexpectedly flake hot 239
cmd/go: "found, but does not contain package" error refers to replaced version instead of its replacement hot 104
net/http: TestRetryRequestsOnError is flaky hot 86
cmd/go: cannot find module providing package error stops `go get` processing hot 63
x/tools/gopls: could not import fmt (no package for import fmt) hot 55
cmd/go: 'cannot find module for path' when importing from subdirectories hot 45
x/tools/gopls: VSCode with go module suddenly shows undeclared name: L LSP[line, character] hot 44
x/net/http2: SetKeepAlivesEnabled(false) closes all HTTP/2 connections older than 5 seconds hot 43
"build constraints exclude all Go files in" when trying to import golang.org/x/sys/unix hot 42
cmd/go: it is inconvenient and unfriendly to require a simple program must have a go.mod file hot 39
x/tools/cmd/gopls: "build cache is required, but could not be located: GOCACHE is not defined and $HOME is not defined" hot 35
encoding/json: invalid use of ,string struct tag, trying to unmarshal "" into int hot 35
x/tools/gopls: support multi-module workspaces hot 30
net/http: TestTransportMaxConnsPerHost flakes hot 30
source：https://uonfu.com/
answerer
Robert Kuska rkuska Czech republic I lift things. And read things.

repos

38

gists

1

followers

17

following

1
Github User Rank List