profile
viewpoint

orange-jacky/albums 13

云图片上传,搜索,下载, 类似于google photos

orange-jacky/gin 1

Gin is a HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster. If you need smashing performance, get yourself some Gin.

orange-jacky/1m-go-websockets 0

handling 1M websockets connections in Go

orange-jacky/apns2 0

⚡ HTTP/2 Apple Push Notification Service (APNs) push provider for Go — Send push notifications to iOS, tvOS, and OSX apps, using the new APNs HTTP/2 protocol.

orange-jacky/atom 0

:atom: The hackable text editor

orange-jacky/bleve 0

A modern text indexing library for go

orange-jacky/bolt 0

An embedded key/value database for Go.

orange-jacky/brew 0

🍺 The missing package manager for macOS

orange-jacky/buford 0

A push notification delivery engine for the new HTTP/2 APNS service.

startedtinygo-org/tinygo

started time in 13 days

fork orange-jacky/examples

chromedp code examples.

fork in a month

issue commentchrislusf/seaweedfs

heartbeat to 9333 19333 failed

when master server hange , cpu is normal

orange-jacky

comment created time in a month

issue commentchrislusf/seaweedfs

heartbeat to 9333 19333 failed

top - 17:52:32 up 171 days, 2:46, 1 user, load average: 0.15, 0.17, 0.12 Tasks: 114 total, 1 running, 79 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.8 us, 0.2 sy, 0.0 ni, 98.8 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st KiB Mem : 3892840 total, 137428 free, 992484 used, 2762928 buff/cache KiB Swap: 969964 total, 969964 free, 0 used. 2700664 avail Mem

orange-jacky

comment created time in a month

issue commentchrislusf/seaweedfs

heartbeat to 9333 19333 failed

[search@iZ8vbf0atmkl998ncf4wx4Z seaweedfs]$./weed version version 30GB 1.71 linux amd64 [search@iZ8vbf0atmkl998ncf4wx4Z seaweedfs]$uname -a Linux iZ8vbf0atmkl998ncf4wx4Z 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux [search@iZ8vbf0atmkl998ncf4wx4Z seaweedfs]$lsb_release -a LSB Version: core-9.20170808ubuntu1-noarch:security-9.20170808ubuntu1-noarch Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic [search@iZ8vbf0atmkl998ncf4wx4Z seaweedfs]$cat master_stdout.log I0408 10:05:16 24272 file_util.go:20] Folder ./mdata Permission: -rwxrwxr-x I0408 10:05:16 24272 master_server.go:104] Volume Size Limit is 30000 MB I0408 10:05:16 24272 master_server.go:192] adminScripts: I0408 10:05:16 24272 master.go:113] Start Seaweed Master 30GB 1.71 at 0.0.0.0:9333 I0408 10:05:16 24272 raft_server.go:47] Starting RaftServer with localhost:9333 I0408 10:05:16 24272 raft_server.go:87] current cluster leader: I0408 10:05:16 24272 master_server.go:147] [ localhost:9333 ] I am the leader! I0408 10:05:16 24272 master.go:137] Start Seaweed Master 30GB 1.71 grpc server at 0.0.0.0:19333 I0408 10:05:16 24272 master_grpc_server.go:252] + client master@127.0.0.1:11858 I0408 10:05:16 24272 master_grpc_server.go:252] + client shell@127.0.0.1:11860 I0408 10:05:16 24272 master_grpc_server.go:263] - client shell@127.0.0.1:11860 I0408 10:05:19 24272 master_server.go:144] state change: &{typ:stateChange source:0xc0001d1200 value:candidate prevValue:follower} I0408 10:05:19 24272 master_server.go:144] state change: &{typ:stateChange source:0xc0001d1200 value:leader prevValue:candidate} I0408 10:05:19 24272 master_server.go:138] event: &{typ:leaderChange source:0xc0001d1200 value:localhost:9333 prevValue:} I0408 10:05:19 24272 master_server.go:140] [ localhost:9333 ] localhost:9333 becomes leader. I0408 10:05:37 24272 node.go:278] topo adds child DefaultDataCenter I0408 10:05:37 24272 node.go:278] topo:DefaultDataCenter adds child DefaultRack I0408 10:05:37 24272 node.go:278] topo:DefaultDataCenter:DefaultRack adds child 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:72] added volume server 127.0.0.1:8080 I0408 10:05:37 24272 volume_layout.go:243] Volume 4 becomes writable I0408 10:05:37 24272 volume_layout.go:243] Volume 2 becomes writable I0408 10:05:37 24272 volume_layout.go:243] Volume 6 becomes writable I0408 10:05:37 24272 volume_layout.go:243] Volume 7 becomes writable I0408 10:05:37 24272 volume_layout.go:243] Volume 5 becomes writable I0408 10:05:37 24272 volume_layout.go:243] Volume 3 becomes writable I0408 10:05:37 24272 volume_layout.go:243] Volume 1 becomes writable I0408 10:05:37 24272 master_grpc_server.go:111] master see new volume 4 from 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:111] master see new volume 2 from 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:111] master see new volume 6 from 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:111] master see new volume 7 from 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:111] master see new volume 5 from 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:111] master see new volume 3 from 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:111] master see new volume 1 from 127.0.0.1:8080 I0408 10:05:37 24272 master_grpc_server.go:157] master send to master@127.0.0.1:11858: url:"127.0.0.1:8080" public_url:"127.0.0.1:8080" new_vids:4 new_vids:2 new_vids:6 new_vids:7 new_vids:5 new_vids:3 new_vids:1 I0408 10:05:53 24272 master_grpc_server.go:252] + client filer@127.0.0.1:18888 SIGQUIT: quit PC=0x467e81 m=0 sigcode=0

goroutine 0 [idle]: runtime.futex(0x2a9d988, 0x80, 0x0, 0x0, 0x32d5d100000000, 0x200000000, 0x2a9cf00, 0x7ffe36779390, 0x7ffe36779368, 0x40b10f, ...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/sys_linux_amd64.s:567 +0x21 runtime.futexsleep(0x2a9d988, 0x0, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/os_linux.go:45 +0x46 runtime.notesleep(0x2a9d988) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/lock_futex.go:151 +0x9f runtime.stopm() /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/proc.go:1828 +0xc0 runtime.exitsyscall0(0xc00002cc00) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/proc.go:3262 +0x111 runtime.mcall(0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/asm_amd64.s:318 +0x5b

goroutine 1 [select (no cases), 14534 minutes]: github.com/chrislusf/seaweedfs/weed/command.startMaster(0xc0000b4ec8, 0xc00004b0c0, 0xc00004b0d0, 0xc00004b0e0, 0xc00004b0f0, 0xc0000b4f58, 0xc0000b4f65, 0xc0000b4f68, 0xc00004b100, 0xc0000b4f70, ...) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/command/master.go:146 +0x11ab github.com/chrislusf/seaweedfs/weed/command.runMaster(0x2a85ac0, 0xc000094170, 0x0, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/command/master.go:99 +0x13f main.main() /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/weed.go:66 +0x2f9

goroutine 18 [chan receive, 4386 minutes]: github.com/chrislusf/seaweedfs/weed/glog.(*loggingT).flushDaemon(0x2a9c360) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:883 +0x8b created by github.com/chrislusf/seaweedfs/weed/glog.init.0 /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:410 +0x26f

goroutine 27 [syscall, 14534 minutes]: os/signal.signal_recv(0x465cf6) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/sigqueue.go:147 +0x9c os/signal.loop() /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/os/signal/signal_unix.go:23 +0x22 created by os/signal.Notify.func1 /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/os/signal/signal.go:127 +0x44

goroutine 9 [chan receive, 14534 minutes]: github.com/chrislusf/seaweedfs/weed/util.init.1.func1() /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/util/signal_handling.go:29 +0x88 created by github.com/chrislusf/seaweedfs/weed/util.init.1 /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/util/signal_handling.go:28 +0x17a

goroutine 10 [select, 4386 minutes]: go.opencensus.io/stats/view.(*worker).start(0xc00007d8b0) /home/travis/gopath/pkg/mod/go.opencensus.io@v0.22.0/stats/view/worker.go:154 +0x100 created by go.opencensus.io/stats/view.init.0 /home/travis/gopath/pkg/mod/go.opencensus.io@v0.22.0/stats/view/worker.go:32 +0x57

goroutine 30 [select, 4386 minutes]: github.com/chrislusf/seaweedfs/weed/stats.(*ServerStats).Start(0xc000510740) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/stats/stats.go:92 +0x273 created by github.com/chrislusf/seaweedfs/weed/server.init.0 /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/common.go:35 +0x57

goroutine 15 [select, 14534 minutes]: google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000459400) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/balancer_conn_wrappers.go:69 +0xc2 created by google.golang.org/grpc.newCCBalancerWrapper /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/balancer_conn_wrappers.go:60 +0x16d

goroutine 32 [sleep, 4386 minutes]: time.Sleep(0x1622bb980) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/time.go:188 +0xba github.com/chrislusf/seaweedfs/weed/topology.(*Topology).StartRefreshWritableVolumes.func1(0xc0001d0d80) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/topology_event_handling.go:19 +0x79 created by github.com/chrislusf/seaweedfs/weed/topology.(*Topology).StartRefreshWritableVolumes /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/topology_event_handling.go:13 +0x43

goroutine 33 [chan receive, 4394 minutes]: github.com/chrislusf/seaweedfs/weed/topology.(*Topology).StartRefreshWritableVolumes.func2(0xc0001d0d80, 0x1d0d2a0, 0xc00000e280, 0x0, 0x3fd3333333333333) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/topology_event_handling.go:24 +0x6c created by github.com/chrislusf/seaweedfs/weed/topology.(*Topology).StartRefreshWritableVolumes /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/topology_event_handling.go:22 +0x8f

goroutine 34 [chan receive, 14534 minutes]: github.com/chrislusf/seaweedfs/weed/topology.(*Topology).StartRefreshWritableVolumes.func3(0xc0001d0d80) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/topology_event_handling.go:33 +0x4a created by github.com/chrislusf/seaweedfs/weed/topology.(*Topology).StartRefreshWritableVolumes /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/topology/topology_event_handling.go:30 +0xb1

goroutine 35 [select, 4386 minutes]: github.com/chrislusf/raft.(*server).leaderLoop(0xc0001d1200) /home/travis/gopath/pkg/mod/github.com/chrislusf/raft@v0.0.0-20190225081310-10d6e2182d92/server.go:847 +0x2f0 github.com/chrislusf/raft.(*server).loop(0xc0001d1200) /home/travis/gopath/pkg/mod/github.com/chrislusf/raft@v0.0.0-20190225081310-10d6e2182d92/server.go:609 +0x19b github.com/chrislusf/raft.(*server).Start.func1(0xc0001d1200) /home/travis/gopath/pkg/mod/github.com/chrislusf/raft@v0.0.0-20190225081310-10d6e2182d92/server.go:470 +0x57 created by github.com/chrislusf/raft.(*server).Start /home/travis/gopath/pkg/mod/github.com/chrislusf/raft@v0.0.0-20190225081310-10d6e2182d92/server.go:468 +0x1a8

goroutine 36 [runnable]: internal/poll.runtime_pollWait(0x7f6d73c75c78, 0x72, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/netpoll.go:203 +0x55 internal/poll.(*pollDesc).wait(0xc000110a18, 0x72, 0x0, 0x0, 0x19b9e5a) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Accept(0xc000110a00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_unix.go:384 +0x1d4 net.(*netFD).accept(0xc000110a00, 0xc00005ea80, 0x7f6d9a8fe108, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/fd_unix.go:238 +0x42 net.(*TCPListener).accept(0xc00000d840, 0x40cf18, 0x30, 0x18e34e0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/tcpsock_posix.go:139 +0x32 net.(*TCPListener).Accept(0xc00000d840, 0xc0004c30b0, 0x2a9bb60, 0x43f15e, 0xc000099e48) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/tcpsock.go:261 +0x64 github.com/chrislusf/seaweedfs/weed/util.(*Listener).Accept(0xc00000d860, 0x1a4f708, 0xc000469800, 0x1d56780, 0xc0004c30b0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/util/net_timeout.go:19 +0x37 google.golang.org/grpc.(*Server).Serve(0xc000469800, 0x1d32f40, 0xc00000d860, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:597 +0x210 created by github.com/chrislusf/seaweedfs/weed/command.startMaster /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/command/master.go:138 +0xd69

goroutine 37 [select, 14534 minutes]: google.golang.org/grpc/internal/transport.(*recvBufferReader).readClient(0xc00045eb40, 0xc000453390, 0x5, 0x5, 0xc0004bf4d8, 0xc0004bf520, 0xc0004bf4d0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:186 +0xd3 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read(0xc00045eb40, 0xc000453390, 0x5, 0x5, 0xc000047060, 0x1a506a8, 0xc0004bf540) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:166 +0x18a google.golang.org/grpc/internal/transport.(*transportReader).Read(0xc0004c2810, 0xc000453390, 0x5, 0x5, 0xc00013b1c8, 0xc0004bf568, 0x8fab3c) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:479 +0x55 io.ReadAtLeast(0x1d0d9e0, 0xc0004c2810, 0xc000453390, 0x5, 0x5, 0x5, 0xc0003b8140, 0xc0004bf610, 0x88e7c6) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 google.golang.org/grpc/internal/transport.(*Stream).Read(0xc0002a4400, 0xc000453390, 0x5, 0x5, 0x0, 0xc0003b8140, 0x18875a0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:463 +0xcd google.golang.org/grpc.(*parser).recvMsg(0xc000453380, 0x40000000, 0x2a7bf40, 0x196b620, 0xc0004c2840, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:508 +0x63 google.golang.org/grpc.recvAndDecompress(0xc000453380, 0xc0002a4400, 0x0, 0x0, 0x40000000, 0x0, 0x0, 0x0, 0xc000042510, 0x29, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:639 +0x4d google.golang.org/grpc.recv(0xc000453380, 0x7f6d73c353a8, 0x2ac8088, 0xc0002a4400, 0x0, 0x0, 0x18875a0, 0xc000487e00, 0x40000000, 0x0, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:707 +0x9b google.golang.org/grpc.(*csAttempt).recvMsg(0xc000184700, 0x18875a0, 0xc000487e00, 0x0, 0xc0004bf980, 0x40c6a6) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:885 +0xed google.golang.org/grpc.(*clientStream).RecvMsg.func1(0xc000184700, 0x203000, 0x203000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:736 +0x46 google.golang.org/grpc.(*clientStream).withRetry(0xc0000c46c0, 0xc0004bf9f8, 0xc0004bf9c8, 0xc0004bf9d0, 0x58) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:590 +0x360 google.golang.org/grpc.(*clientStream).RecvMsg(0xc0000c46c0, 0x18875a0, 0xc000487e00, 0x165b280, 0xc0001104d0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:735 +0x103 github.com/chrislusf/seaweedfs/weed/pb/master_pb.(*seaweedKeepConnectedClient).Recv(0xc0002e0850, 0xc000000001, 0xc000045cd0, 0xe) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/master_pb/master.pb.go:1609 +0x62 github.com/chrislusf/seaweedfs/weed/wdclient.(*MasterClient).tryConnectToMaster.func1(0x1d5a700, 0xc0000b8108, 0xc0000b80e8, 0xc0000441f8) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/wdclient/masterclient.go:86 +0x912 github.com/chrislusf/seaweedfs/weed/pb.WithMasterClient.func1(0xc000132e00, 0xc000270900, 0xc0000441e0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/grpc_client_server.go:156 +0x6b github.com/chrislusf/seaweedfs/weed/pb.WithCachedGrpcClient(0xc0004bfdd8, 0xc0000441e0, 0xf, 0xc00009ade8, 0x1, 0x1, 0x7f6d9a8fe108, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/grpc_client_server.go:104 +0x3d4 github.com/chrislusf/seaweedfs/weed/pb.WithMasterClient(0xc0000b5c70, 0xe, 0x1d0d2a0, 0xc00000e280, 0xc00009aea0, 0x203000, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/grpc_client_server.go:154 +0x1ae github.com/chrislusf/seaweedfs/weed/wdclient.(*MasterClient).tryConnectToMaster(0xc000110480, 0xc0000b5c70, 0xe, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/wdclient/masterclient.go:69 +0x165 github.com/chrislusf/seaweedfs/weed/wdclient.(*MasterClient).tryAllMasters(0xc000110480) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/wdclient/masterclient.go:57 +0x10d github.com/chrislusf/seaweedfs/weed/wdclient.(*MasterClient).KeepConnectedToMaster(0xc000110480) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/wdclient/masterclient.go:48 +0xd1 created by github.com/chrislusf/seaweedfs/weed/command.startMaster /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/command/master.go:140 +0xd92

goroutine 38 [IO wait, 14534 minutes]: internal/poll.runtime_pollWait(0x7f6d73c75d58, 0x72, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/netpoll.go:203 +0x55 internal/poll.(*pollDesc).wait(0xc000110818, 0x72, 0x0, 0x0, 0x19b9e5a) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Accept(0xc000110800, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_unix.go:384 +0x1d4 net.(*netFD).accept(0xc000110800, 0x182c860, 0xc00002be38, 0x40c6a6) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/fd_unix.go:238 +0x42 net.(*TCPListener).accept(0xc00000d040, 0x2a9d840, 0x30, 0x30) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/tcpsock_posix.go:139 +0x32 net.(*TCPListener).Accept(0xc00000d040, 0x0, 0x50, 0x182c860, 0xa0939247eda1c101) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/tcpsock.go:261 +0x64 github.com/chrislusf/seaweedfs/weed/util.(*Listener).Accept(0xc00000d060, 0x188b740, 0xc000023710, 0x16ede00, 0x2a766a0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/util/net_timeout.go:19 +0x37 net/http.(*Server).Serve(0xc0002bc0e0, 0x1d32f40, 0xc00000d060, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/http/server.go:2901 +0x25d created by github.com/chrislusf/seaweedfs/weed/command.startMaster /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/command/master.go:144 +0x11a6

goroutine 16 [chan receive, 14534 minutes]: google.golang.org/grpc.(*addrConn).resetTransport(0xc0001d22c0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/clientconn.go:1164 +0x6ea created by google.golang.org/grpc.(*addrConn).connect /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/clientconn.go:800 +0x128

goroutine 52 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*http2Client).keepalive(0xc00010e000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:1330 +0x193 created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:295 +0x16cd

goroutine 53 [IO wait, 4386 minutes]: internal/poll.runtime_pollWait(0x7f6d73c75b98, 0x72, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/netpoll.go:203 +0x55 internal/poll.(*pollDesc).wait(0xc000184d98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0xc000184d80, 0xc0004e8000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_unix.go:169 +0x19b net.(*netFD).Read(0xc000184d80, 0xc0004e8000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc0000b8170, 0xc0004e8000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/net.go:184 +0x8e bufio.(*Reader).Read(0xc000487560, 0xc0001041f8, 0x9, 0x9, 0xc00009ae08, 0x405643, 0xc000048cc0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/bufio/bufio.go:226 +0x24f io.ReadAtLeast(0x1d09fc0, 0xc000487560, 0xc0001041f8, 0x9, 0x9, 0x9, 0x45fd90, 0xc000048cc0, 0xc00009ae40) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc0001041f8, 0x9, 0x9, 0x1d09fc0, 0xc000487560, 0x0, 0xc000000000, 0x10000000040cf18, 0xc00045ea10) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:237 +0x87 golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001041c0, 0xc00001d240, 0x2a9bb60, 0x0, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:492 +0xa1 google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc00010e000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:1264 +0x16f created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:300 +0xd23

goroutine 43 [select, 4386 minutes]: github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).KeepConnected(0xc000115b20, 0x1d56600, 0xc0002fbf30, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/master_grpc_server.go:218 +0x3dd github.com/chrislusf/seaweedfs/weed/pb/master_pb._Seaweed_KeepConnected_Handler(0x1981600, 0xc000115b20, 0x1d4f460, 0xc0000180c0, 0x2ac8088, 0xc0000f4a00) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/master_pb/master.pb.go:1743 +0xad google.golang.org/grpc.(*Server).processStreamingRPC(0xc000469800, 0x1d58100, 0xc00002c180, 0xc0000f4a00, 0xc000023500, 0x2a80960, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1237 +0xcd1 google.golang.org/grpc.(*Server).handleStream(0xc000469800, 0x1d58100, 0xc00002c180, 0xc0000f4a00, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1317 +0xcd6 google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc00001bc20, 0xc000469800, 0x1d58100, 0xc00002c180, 0xc0000f4a00) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722 +0xa1 created by google.golang.org/grpc.(*Server).serveStreams.func1 /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:720 +0xa1

goroutine 54 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00045ea00, 0x1, 0x0, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:395 +0x122 google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000487620, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:513 +0x1cd google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00010e000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:346 +0x7b created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:344 +0xedb

goroutine 40 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0000d4460, 0x1, 0x0, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:395 +0x122 google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000575da0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:513 +0x1cd google.golang.org/grpc/internal/transport.newHTTP2Server.func2(0xc00002c180) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:296 +0xcb created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:293 +0x10c1

goroutine 41 [runnable]: runtime.resettimer(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/time.go:488 time.resetTimer(0xc0000d45a8, 0x3384e3ab8f5216) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/time.go:223 +0x62 time.(*Timer).Reset(0xc0000d45a0, 0xffffffffffec2204, 0x2a9bb60) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/time/sleep.go:126 +0x7d google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc00002c180) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:1004 +0x6ba created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:302 +0x10e6

goroutine 42 [IO wait, 4386 minutes]: internal/poll.runtime_pollWait(0x7f6d73c75ab8, 0x72, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/netpoll.go:203 +0x55 internal/poll.(*pollDesc).wait(0xc000110a98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0xc000110a80, 0xc000116000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_unix.go:169 +0x19b net.(*netFD).Read(0xc000110a80, 0xc000116000, 0x8000, 0x8000, 0x405125, 0xc0000bb320, 0xc000315ca0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc00000e490, 0xc000116000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/net.go:184 +0x8e github.com/chrislusf/seaweedfs/weed/util.(*Conn).Read(0xc0000237d0, 0xc000116000, 0x8000, 0x8000, 0x20, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/util/net_timeout.go:48 +0x6c bufio.(*Reader).Read(0xc000575d40, 0xc0002bc2d8, 0x9, 0x9, 0xc00005f500, 0x7f6d9a8fe7d0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/bufio/bufio.go:226 +0x24f io.ReadAtLeast(0x1d09fc0, 0xc000575d40, 0xc0002bc2d8, 0x9, 0x9, 0x9, 0x8ce155, 0xc00001d26c, 0xc0001805f0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc0002bc2d8, 0x9, 0x9, 0x1d09fc0, 0xc000575d40, 0x0, 0x0, 0xc00001d260, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:237 +0x87 golang.org/x/net/http2.(*Framer).ReadFrame(0xc0002bc2a0, 0xc00001d260, 0x2a9bb60, 0x0, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:492 +0xa1 google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc00002c180, 0xc000023920, 0x1a4f748) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:458 +0x98 google.golang.org/grpc.(*Server).serveStreams(0xc000469800, 0x1d58100, 0xc00002c180) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:718 +0xdb google.golang.org/grpc.(*Server).handleRawConn.func1(0xc000469800, 0x1d58100, 0xc00002c180) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:679 +0x3f created by google.golang.org/grpc.(*Server).handleRawConn /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:678 +0x562

goroutine 55 [select, 14534 minutes]: google.golang.org/grpc.newClientStream.func5(0xc000132e00, 0xc0000c46c0, 0x1d430c0, 0xc000271e90) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:319 +0xd7 created by google.golang.org/grpc.newClientStream /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:318 +0xbbf

goroutine 44 [select, 14534 minutes]: google.golang.org/grpc/internal/transport.(*recvBufferReader).read(0xc0000d46e0, 0xc00000dc70, 0x5, 0x5, 0x0, 0x1, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:174 +0xc7 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read(0xc0000d46e0, 0xc00000dc70, 0x5, 0x5, 0xc00001c900, 0x1a506a8, 0xc00009cb50) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:168 +0x20b google.golang.org/grpc/internal/transport.(*transportReader).Read(0xc000023bf0, 0xc00000dc70, 0x5, 0x5, 0xc0002fbf30, 0xc00009cb78, 0x8fbf2c) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:479 +0x55 io.ReadAtLeast(0x1d0d9e0, 0xc000023bf0, 0xc00000dc70, 0x5, 0x5, 0x5, 0xc00005ea80, 0x7f6d9a8fe7d0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 google.golang.org/grpc/internal/transport.(*Stream).Read(0xc0000f4a00, 0xc00000dc70, 0x5, 0x5, 0x30, 0x188b740, 0x19b5e01) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:463 +0xcd google.golang.org/grpc.(*parser).recvMsg(0xc00000dc60, 0x40000000, 0x1963200, 0xc0000f4a00, 0xc0004b6530, 0x40c6a6, 0xc00014a8c0, 0x70) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:508 +0x63 google.golang.org/grpc.recvAndDecompress(0xc00000dc60, 0xc0000f4a00, 0x0, 0x0, 0x40000000, 0x0, 0x0, 0x0, 0x0, 0x19b96a4, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:639 +0x4d google.golang.org/grpc.recv(0xc00000dc60, 0x7f6d73c353a8, 0x2ac8088, 0xc0000f4a00, 0x0, 0x0, 0x18026e0, 0xc0000aa060, 0x40000000, 0x0, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:707 +0x9b google.golang.org/grpc.(*serverStream).RecvMsg(0xc0000180c0, 0x18026e0, 0xc0000aa060, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:1474 +0x156 github.com/chrislusf/seaweedfs/weed/pb/master_pb.(*seaweedKeepConnectedServer).Recv(0xc0002fbf30, 0xc00045eb90, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/master_pb/master.pb.go:1762 +0x62 github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).KeepConnected.func1(0x1d56600, 0xc0002fbf30, 0xc00001c920, 0x16, 0xc000090f60) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/master_grpc_server.go:207 +0x3b created by github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).KeepConnected /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/master_grpc_server.go:205 +0x2ea

goroutine 59 [runnable]: internal/poll.runtime_pollWait(0x7f6d73c758f8, 0x72, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/netpoll.go:203 +0x55 internal/poll.(*pollDesc).wait(0xc000185e98, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0xc000185e80, 0xc0003c6000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_unix.go:169 +0x19b net.(*netFD).Read(0xc000185e80, 0xc0003c6000, 0x8000, 0x8000, 0x405125, 0xc0000bb320, 0xc000029ca0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc0000b82a0, 0xc0003c6000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/net.go:184 +0x8e github.com/chrislusf/seaweedfs/weed/util.(*Conn).Read(0xc0004c30b0, 0xc0003c6000, 0x8000, 0x8000, 0x20, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/util/net_timeout.go:48 +0x6c bufio.(*Reader).Read(0xc000487f20, 0xc0001042d8, 0x9, 0x9, 0xc00005f500, 0x7f6d9a8fe108, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/bufio/bufio.go:226 +0x24f io.ReadAtLeast(0x1d09fc0, 0xc000487f20, 0xc0001042d8, 0x9, 0x9, 0x9, 0x8ce155, 0xc00014528c, 0xc000049020) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc0001042d8, 0x9, 0x9, 0x1d09fc0, 0xc000487f20, 0x0, 0x0, 0xc000145280, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:237 +0x87 golang.org/x/net/http2.(*Framer).ReadFrame(0xc0001042a0, 0xc000145280, 0x2a9bb60, 0x0, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:492 +0xa1 google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc0000bd800, 0xc0004c3200, 0x1a4f748) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:458 +0x98 google.golang.org/grpc.(*Server).serveStreams(0xc000469800, 0x1d58100, 0xc0000bd800) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:718 +0xdb google.golang.org/grpc.(*Server).handleRawConn.func1(0xc000469800, 0x1d58100, 0xc0000bd800) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:679 +0x3f created by google.golang.org/grpc.(*Server).handleRawConn /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:678 +0x562

goroutine 57 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0003be9b0, 0x1, 0x0, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:395 +0x122 google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc0003e6060, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:513 +0x1cd google.golang.org/grpc/internal/transport.newHTTP2Server.func2(0xc0000bd800) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:296 +0xcb created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:293 +0x10c1

goroutine 58 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc0000bd800) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:968 +0x285 created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:302 +0x10e6

goroutine 72 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*recvBufferReader).read(0xc0000d5180, 0xc0000aab30, 0x5, 0x5, 0xc0003b5240, 0x88cc15, 0xc00000c800) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:174 +0xc7 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read(0xc0000d5180, 0xc0000aab30, 0x5, 0x5, 0xc00001ca80, 0x1a506a8, 0xc0003b5230) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:168 +0x20b google.golang.org/grpc/internal/transport.(*transportReader).Read(0xc000320810, 0xc0000aab30, 0x5, 0x5, 0x165b5c0, 0xc0003b5258, 0x8fbf2c) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:479 +0x55 io.ReadAtLeast(0x1d0d9e0, 0xc000320810, 0xc0000aab30, 0x5, 0x5, 0x5, 0x8, 0x196b620, 0xc0004c2840) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 google.golang.org/grpc/internal/transport.(*Stream).Read(0xc0000f5000, 0xc0000aab30, 0x5, 0x5, 0x17d0e80, 0x0, 0xc000325a50) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:463 +0xcd google.golang.org/grpc.(*parser).recvMsg(0xc0000aab20, 0x40000000, 0x40cf18, 0x20, 0x17d0e80, 0x7f6d9a8fe101, 0xc00000c820, 0xc0003b53d8) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:508 +0x63 google.golang.org/grpc.recvAndDecompress(0xc0000aab20, 0xc0000f5000, 0x0, 0x0, 0x40000000, 0x0, 0x0, 0x0, 0x948ce9, 0xc000018240, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:639 +0x4d google.golang.org/grpc.recv(0xc0000aab20, 0x7f6d73c353a8, 0x2ac8088, 0xc0000f5000, 0x0, 0x0, 0x193aee0, 0xc000010e00, 0x40000000, 0x0, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:707 +0x9b google.golang.org/grpc.(*serverStream).RecvMsg(0xc000018240, 0x193aee0, 0xc000010e00, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:1474 +0x156 github.com/chrislusf/seaweedfs/weed/pb/master_pb.(*seaweedSendHeartbeatServer).Recv(0xc000322290, 0xc0000d4cd0, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/master_pb/master.pb.go:1736 +0x62 github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).SendHeartbeat(0xc000115b20, 0x1d566c0, 0xc000322290, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/master_grpc_server.go:53 +0x10d3 github.com/chrislusf/seaweedfs/weed/pb/master_pb._Seaweed_SendHeartbeat_Handler(0x1981600, 0xc000115b20, 0x1d4f460, 0xc000018240, 0x2ac8088, 0xc0000f5000) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/master_pb/master.pb.go:1717 +0xad google.golang.org/grpc.(*Server).processStreamingRPC(0xc000469800, 0x1d58100, 0xc00002d380, 0xc0000f5000, 0xc000023500, 0x2a80940, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1237 +0xcd1 google.golang.org/grpc.(*Server).handleStream(0xc000469800, 0x1d58100, 0xc00002d380, 0xc0000f5000, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1317 +0xcd6 google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000324090, 0xc000469800, 0x1d58100, 0xc00002d380, 0xc0000f5000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722 +0xa1 created by google.golang.org/grpc.(*Server).serveStreams.func1 /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:720 +0xa1

goroutine 69 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc0000d4eb0, 0x1, 0x0, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:395 +0x122 google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc00002ec00, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:513 +0x1cd google.golang.org/grpc/internal/transport.newHTTP2Server.func2(0xc00002d380) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:296 +0xcb created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:293 +0x10c1

goroutine 70 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*http2Server).keepalive(0xc00002d380) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:968 +0x285 created by google.golang.org/grpc/internal/transport.newHTTP2Server /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:302 +0x10e6

goroutine 71 [runnable]: internal/poll.runtime_pollWait(0x7f6d73c759d8, 0x72, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/netpoll.go:203 +0x55 internal/poll.(*pollDesc).wait(0xc000110d18, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0xc000110d00, 0xc000340000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_unix.go:169 +0x19b net.(*netFD).Read(0xc000110d00, 0xc000340000, 0x8000, 0x8000, 0x405125, 0xc0000bb320, 0xc00009fca0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc00000e4c0, 0xc000340000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/net.go:184 +0x8e github.com/chrislusf/seaweedfs/weed/util.(*Conn).Read(0xc000320450, 0xc000340000, 0x8000, 0x8000, 0x20, 0xc00009fd10, 0x18) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/util/net_timeout.go:48 +0x6c bufio.(*Reader).Read(0xc00002eba0, 0xc0002bc498, 0x9, 0x9, 0xc000161c00, 0x1007f6d9a8fe7d0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/bufio/bufio.go:226 +0x24f io.ReadAtLeast(0x1d09fc0, 0xc00002eba0, 0xc0002bc498, 0x9, 0x9, 0x9, 0x5e967c1d, 0xc00009fdc8, 0x4c99e6) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc0002bc498, 0x9, 0x9, 0x1d09fc0, 0xc00002eba0, 0x0, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:237 +0x87 golang.org/x/net/http2.(*Framer).ReadFrame(0xc0002bc460, 0xc00001d420, 0x2a9bb60, 0x0, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:492 +0xa1 google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams(0xc00002d380, 0xc0003205a0, 0x1a4f748) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_server.go:458 +0x98 google.golang.org/grpc.(*Server).serveStreams(0xc000469800, 0x1d58100, 0xc00002d380) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:718 +0xdb google.golang.org/grpc.(*Server).handleRawConn.func1(0xc000469800, 0x1d58100, 0xc00002d380) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:679 +0x3f created by google.golang.org/grpc.(*Server).handleRawConn /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:678 +0x562

goroutine 73 [select, 4386 minutes]: github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).KeepConnected(0xc000115b20, 0x1d56600, 0xc00035e0d0, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/master_grpc_server.go:218 +0x3dd github.com/chrislusf/seaweedfs/weed/pb/master_pb._Seaweed_KeepConnected_Handler(0x1981600, 0xc000115b20, 0x1d4f460, 0xc000524000, 0x2ac8088, 0xc0000f4000) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/master_pb/master.pb.go:1743 +0xad google.golang.org/grpc.(*Server).processStreamingRPC(0xc000469800, 0x1d58100, 0xc0000bd800, 0xc0000f4000, 0xc000023500, 0x2a80960, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1237 +0xcd1 google.golang.org/grpc.(*Server).handleStream(0xc000469800, 0x1d58100, 0xc0000bd800, 0xc0000f4000, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:1317 +0xcd6 google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0003c47e0, 0xc000469800, 0x1d58100, 0xc0000bd800, 0xc0000f4000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:722 +0xa1 created by google.golang.org/grpc.(*Server).serveStreams.func1 /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/server.go:720 +0xa1

goroutine 74 [select, 14533 minutes]: google.golang.org/grpc/internal/transport.(*recvBufferReader).read(0xc00045e0a0, 0xc000452630, 0x5, 0x5, 0xc0002ff910, 0x2, 0x2) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:174 +0xc7 google.golang.org/grpc/internal/transport.(*recvBufferReader).Read(0xc00045e0a0, 0xc000452630, 0x5, 0x5, 0xc0000b62c0, 0x1a506a8, 0xc000356b50) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:168 +0x20b google.golang.org/grpc/internal/transport.(*transportReader).Read(0xc000175d40, 0xc000452630, 0x5, 0x5, 0xc00035e0d0, 0xc000356b78, 0x8fbf2c) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:479 +0x55 io.ReadAtLeast(0x1d0d9e0, 0xc000175d40, 0xc000452630, 0x5, 0x5, 0x5, 0x2a9d840, 0x7f6d9a8fe7d0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 google.golang.org/grpc/internal/transport.(*Stream).Read(0xc0000f4000, 0xc000452630, 0x5, 0x5, 0x30, 0x188b740, 0x1) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/transport.go:463 +0xcd google.golang.org/grpc.(*parser).recvMsg(0xc000452620, 0x40000000, 0x1963200, 0xc0000f4000, 0x0, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:508 +0x63 google.golang.org/grpc.recvAndDecompress(0xc000452620, 0xc0000f4000, 0x0, 0x0, 0x40000000, 0x0, 0x0, 0x0, 0x0, 0x0, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:639 +0x4d google.golang.org/grpc.recv(0xc000452620, 0x7f6d73c353a8, 0x2ac8088, 0xc0000f4000, 0x0, 0x0, 0x18026e0, 0xc000452900, 0x40000000, 0x0, ...) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/rpc_util.go:707 +0x9b google.golang.org/grpc.(*serverStream).RecvMsg(0xc000524000, 0x18026e0, 0xc000452900, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/stream.go:1474 +0x156 github.com/chrislusf/seaweedfs/weed/pb/master_pb.(*seaweedKeepConnectedServer).Recv(0xc00035e0d0, 0x0, 0x0, 0x0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/pb/master_pb/master.pb.go:1762 +0x62 github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).KeepConnected.func1(0x1d56600, 0xc00035e0d0, 0xc0000b62e0, 0x15, 0xc0000901e0) /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/master_grpc_server.go:207 +0x3b created by github.com/chrislusf/seaweedfs/weed/server.(*MasterServer).KeepConnected /home/travis/gopath/src/github.com/chrislusf/seaweedfs/weed/server/master_grpc_server.go:205 +0x2ea

goroutine 76 [select, 14519 minutes]: google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc000458a40) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/balancer_conn_wrappers.go:69 +0xc2 created by google.golang.org/grpc.newCCBalancerWrapper /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/balancer_conn_wrappers.go:60 +0x16d

goroutine 77 [chan receive, 14519 minutes]: google.golang.org/grpc.(*addrConn).resetTransport(0xc0001d2000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/clientconn.go:1164 +0x6ea created by google.golang.org/grpc.(*addrConn).connect /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/clientconn.go:800 +0x128

goroutine 79 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*http2Client).keepalive(0xc0002be000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:1330 +0x193 created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:295 +0x16cd

goroutine 80 [runnable]: internal/poll.runtime_pollWait(0x7f6d73c75818, 0x72, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/runtime/netpoll.go:203 +0x55 internal/poll.(*pollDesc).wait(0xc000185a18, 0x72, 0x8000, 0x8000, 0xffffffffffffffff) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:87 +0x45 internal/poll.(*pollDesc).waitRead(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_poll_runtime.go:92 internal/poll.(*FD).Read(0xc000185a00, 0xc000318000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/internal/poll/fd_unix.go:169 +0x19b net.(*netFD).Read(0xc000185a00, 0xc000318000, 0x8000, 0x8000, 0x800000601, 0x0, 0x60100000000) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/fd_unix.go:202 +0x4f net.(*conn).Read(0xc0000b8118, 0xc000318000, 0x8000, 0x8000, 0x0, 0x0, 0x0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/net/net.go:184 +0x8e bufio.(*Reader).Read(0xc000574ba0, 0xc0002bc038, 0x9, 0x9, 0xc000025e08, 0x405643, 0xc0000911a0) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/bufio/bufio.go:226 +0x24f io.ReadAtLeast(0x1d09fc0, 0xc000574ba0, 0xc0002bc038, 0x9, 0x9, 0x9, 0x45fd90, 0xc0000911a0, 0xc000025e40) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:310 +0x87 io.ReadFull(...) /home/travis/.gimme/versions/go1.14.1.linux.amd64/src/io/io.go:329 golang.org/x/net/http2.readFrameHeader(0xc0002bc038, 0x9, 0x9, 0x1d09fc0, 0xc000574ba0, 0x0, 0xc000000000, 0x10000000040cf18, 0xc00007c470) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:237 +0x87 golang.org/x/net/http2.(*Framer).ReadFrame(0xc0002bc000, 0xc00001d3e0, 0x2a9bb60, 0x0, 0x0) /home/travis/gopath/pkg/mod/golang.org/x/net@v0.0.0-20190909003024-a7b16738d86b/http2/frame.go:492 +0xa1 google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc0002be000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:1264 +0x16f created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:300 +0xd23

goroutine 81 [select, 4386 minutes]: google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00007c460, 0x1, 0x0, 0x0, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:395 +0x122 google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc000574d20, 0x0, 0x0) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/controlbuf.go:513 +0x1cd google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc0002be000) /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:346 +0x7b created by google.golang.org/grpc/internal/transport.newHTTP2Client /home/travis/gopath/pkg/mod/google.golang.org/grpc@v1.26.0/internal/transport/http2_client.go:344 +0xedb

rax 0xca rbx 0x2a9d840 rcx 0x467e83 rdx 0x0 rdi 0x2a9d988 rsi 0x80 rbp 0x7ffe36779330 rsp 0x7ffe367792e8 r8 0x0 r9 0x0 r10 0x0 r11 0x286 r12 0x0 r13 0x0 r14 0x0 r15 0x0 rip 0x467e81 rflags 0x286 cs 0x33 fs 0x0 gs 0x0 [search@iZ8vbf0atmkl998ncf4wx4Z seaweedfs]$

orange-jacky

comment created time in a month

issue commentchrislusf/seaweedfs

heartbeat to 9333 19333 failed

i think master server could fake die, and not work, it's a bug

orange-jacky

comment created time in a month

issue commentchrislusf/seaweedfs

heartbeat to 9333 19333 failed

how to repoen this issue

orange-jacky

comment created time in a month

issue commentchrislusf/seaweedfs

heartbeat to 9333 19333 failed

master server, volume server, filer installed on same host, use default ip config not possible network problem, if you think network problem, how to install master server, volume server, filer on same host

there is not firewall

orange-jacky

comment created time in a month

issue openedchrislusf/seaweedfs

heartbeat to 9333 19333 failed

i run weed 1.7 on ubuntu 18.04

my install step: 1.start master server ./weed master -mdir="./mdata" >master_stdout.log 2>&1 & 2.start vloume server ./weed volume -max=100 -mserver="localhost:9333" -dir="./data" >volume_stdout.log 2>&1 &

3.start filer ./weed scaffold -config=filer -output="." vi filer.toml

[leveldb] enabled = true dir = "./leveldb" # directory to store level db files

mkdir leveldb

./weed filer > filer_stdout.log 2>&1 &

weed not work after running some days. and i can't find why.

i upload master server, volume server ,filer output log for you, hope fix this issue for

log attachment filer_stdout.log.zip master_stdout.log.zip volume_stdout.log.zip

created time in a month

issue commentchrislusf/seaweedfs

filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout

@orange-jacky your port 19333 is open through Firewall or not? if not then allows its connection to port 9333 and 9333's to 19333.

master, volume, filer run the same host, not through firewall, port 9333 and 19333 still stay, and telent localhost 9333, it works.

[ubuntu@VM-0-3-ubuntu seaweedfs]$ss -tnlp | grep weed LISTEN 0 128 *:8080 : users:(("weed",pid=10505,fd=13))
LISTEN 0 128 *:9333 : users:(("weed",pid=10505,fd=9))
LISTEN 0 128 *:8888 : users:(("weed",pid=10649,fd=42))
LISTEN 0 128 *:18080 : users:(("weed",pid=10505,fd=12))
LISTEN 0 128 *:19333 : users:(("weed",pid=10505,fd=11))
LISTEN 0 128 *:18888 : users:(("weed",pid=10649,fd=43))

[ubuntu@VM-0-3-ubuntu seaweedfs]$telnet localhost 9333 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. ^C Connection closed by foreign host. [ubuntu@VM-0-3-ubuntu seaweedfs]$telnet localhost 19333 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. @^C

and filer still ouput rpc localhost:9333 err. it's strange

orange-jacky

comment created time in 2 months

issue commentchrislusf/seaweedfs

filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout

I0402 22:48:49 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:48:49 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:51:07 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:51:07 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:53:37 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:53:37 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:53:54 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:53:54 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:54:26 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:54:26 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:54:32 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:54:32 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:58:23 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 22:58:23 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:03:40 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:03:40 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:05:15 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:05:15 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:08:06 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:08:06 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:23:56 10649 masterclient.go:88] filer failed to receive from localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing I0402 23:23:56 10649 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Canceled desc = grpc: the client connection is closing

orange-jacky

comment created time in 2 months

issue openedchrislusf/seaweedfs

filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout

Describe the bug I0402 13:59:50 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 13:59:50 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:00:11 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:00:11 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:00:32 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:00:32 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:00:53 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:00:53 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:01:14 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:01:14 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout"

I0402 14:01:35 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:01:35 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:01:56 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:01:56 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:02:17 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:02:17 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:02:38 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:02:38 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:02:59 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:02:59 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:03:20 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:03:20 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:03:41 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:03:41 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:04:02 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:04:02 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:04:23 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:04:23 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:04:44 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:04:44 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:05:05 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:05:05 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:05:26 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:05:26 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:05:47 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:05:47 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:06:08 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:06:08 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:06:29 15839 masterclient.go:73] filer failed to keep connected to localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout" I0402 14:06:29 15839 masterclient.go:116] filer failed to connect with master localhost:9333: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp 127.0.0.1:19333: i/o timeout"

System Setup

  • List the command line to start "weed master", "weed volume", "weed filer", "weed s3", "weed mount".
  • OS version
  • output of weed version
  • if using filer, show the content of filer.toml ./weed server -master.port=9333 -volume.port=8080 -dir="./data"

./weed scaffold -config=filer -output="." vi filer.toml [leveldb] enabled = true dir = "./leveldb" # directory to store level db files

./weed filer > filer_stdout.log 2>&1 &

created time in 2 months

startedgolang/go

started time in 2 months

startedShopify/sarama

started time in 2 months

startedpion/webrtc

started time in 3 months

startedchrislusf/seaweedfs

started time in 3 months

startedmicro/go-micro

started time in 3 months

issue openedgolang/go

how to close tcp keep-alive for http.ListenAndServer

<!-- Please answer these questions before submitting your issue. Thanks! -->

What version of Go are you using (go version)?

<pre> $ go version go version go1.14 darwin/amd64 </pre>

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env GO111MODULE="on" GOARCH="amd64" GOBIN="/Users/xx/Documents/develop/go/workspace/bin" GOCACHE="/Users/xx/Library/Caches/go-build" GOENV="/Users/xx/Library/Application Support/go/env" GOEXE="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="darwin" GOINSECURE="" GONOPROXY="" GONOSUMDB="" GOOS="darwin" GOPATH="/Users/xx/Documents/develop/go/workspace" GOPRIVATE="" GOPROXY="direct" GOROOT="/usr/local/go" GOSUMDB="off" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64" GCCGO="gccgo" AR="ar" CC="clang" CXX="clang++" CGO_ENABLED="1" GOMOD="/dev/null" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/7r/fpcvbc6d32j2lfls0lllcs240000gn/T/go-build785679791=/tmp/go-build -gno-record-gcc-switches -fno-common" GOROOT/bin/go version: go version go1.14 darwin/amd64 GOROOT/bin/go tool compile -V: compile version go1.14 uname -v: Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 ProductName: Mac OS X ProductVersion: 10.15.3 BuildVersion: 19D76 lldb --version: lldb-1100.0.30.12 Apple Swift version 5.1.3 (swiftlang-1100.0.282.1 clang-1100.0.33.15) </pre></details>

What did you do?

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

edit main.go

`package main

import ( "log" "net/http" )

func main() { addr := "127.0.0.1:8080" http.HandleFunc("/agent", agentHandler)

srv := http.Server{
	Addr: addr,
}
srv.SetKeepAlivesEnabled(false)

err := srv.ListenAndServe()
log.Fatal(err)

}

func agentHandler(w http.ResponseWriter, r *http.Request) { } `

in terminal1 go run main.go

in terminal2 telnet 127.0.0.1 8080

What did you expect to see?

http server not actively send a tcp keep-alive to client or i can change time of send tcp keep-alive

What did you see instead?

every 15s, http server actively send a tcp keep-alive to client

image

created time in 3 months

more