google/go-cmp 2108
Package for comparing Go values in tests
Safe Browsing API Go Client
dsnet/compress 319
Collection of compression related Go packages.
Daemon for creating a simple VPN over UDP.
Custom message-of-the-day (MOTD) generator intended to be informative about the system you are logging in to.
SSH daemon for creating forward and reverse tunnels.
dsnet/golib 16
Collection of mostly unrelated helper Go packages.
Locally hosted Go playground for more advanced functionality.
TermiJack surreptitiously hijacks standard streams (stdin, stdout, and/or stderr) from an already running process.
dsnet/gotab 11
Simple bash tab completion for the go command.
PR opened gogo/protobuf
In some scenarios, we need to render number64 type such as int64, uint64 as integers instead of string values, but found we cannot. So I add an option in config to fix this.
pr created time in 20 hours
startedyuppity/unifi-video-api
started time in 21 hours
issue closedgolang/protobuf
How to get a map's value's golang package and golang name when writing a plugin
test/api/test.proto
syntax="proto3";
package test;
option go_package="test/api;api";
import "test/first/first.proto";
service test{
rpc hello(hreq)returns(hresp);
}
message hreq{
map<int32,first.a.abc> test=1;
}
message hresp{
string w=1;
}
test/first/first.proto
syntax="proto3";
package first;
option go_package="test/first;first";
message a{
enum abc{
A = 0;
B = 1;
C = 2;
}
}
when generate the test/api/test.proto
how to get the first.a.abc
go pkg and go name
the first.a.abc
can change to a message type,how to get the go package and go name
closed time in a day
chenjie199234issue openedgolang/protobuf
How to get a map's value's golang package and golang name when writing a plugin
test/api/test.proto
syntax="proto3";
package test;
option go_package="test/api;api";
import "test/first/first.proto";
service test{
rpc hello(hreq)returns(hresp);
}
message hreq{
map<int32,first.a.abc> test=1;
}
message hresp{
string w=1;
}
test/first/first.proto
syntax="proto3";
package first;
option go_package="test/first;first";
message a{
enum abc{
A = 0;
B = 1;
C = 2;
}
}
when generate the test/api/test.proto
how to get the first.a.abc
go pkg and go name
created time in a day
startednorvig/pytudes
started time in 2 days
issue commentgolang/protobuf
String and protojson produces non-deterministic whitespaces.
Thanks a lot, sorry for duplicate!
comment created time in 3 days
issue openedgolang/protobuf
String and protojson produces nondeterministic whitespaces.
What version of protobuf and what language are you using?
github.com/golang/protobuf v1.4.3
github.com/opentracing/opentracing-go v1.1.0
github.com/pkg/errors v0.9.1
github.com/stretchr/testify v1.6.1
golang.org/x/net v0.0.0-20190620200207-3b0461eec859
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
google.golang.org/grpc v1.30.1
google.golang.org/grpc/examples v0.0.0-20200723182653-9106c3fff523
google.golang.org/protobuf v1.25.0
protoc-gen-go: require google.golang.org/protobuf v1.25.0 // cmd/protoc-gen-go
(We use bingo for tool pinning)
Go 1.15
What did you do?
Okay, it's extremely weird so stay with me. (:
The repro test is here https://github.com/grpc-ecosystem/go-grpc-middleware/blob/repro/grpctesting/testpb/test_repro_test.go you can find proto as well.
Now it's tricky to reproduce, because it depends on... code line you invoke String or protojson? Like some compiler optimization is changing things? I know it's super weird, but it looks like this:
Steps to repro (I am using Goland test invocation):
- Run test.
package testpb
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/encoding/protojson"
)
func TestNonDeterministicMarshal(t *testing.T) {
p := &PingRequest{Value: "something", ErrorCodeReturned: 4}
// Remove this line to see the test failing (:
// Sometimes
// * "value:\"something\" error_code_returned:4"
// * "value:\"something\" error_code_returned:4"
assert.Equal(t, "value:\"something\" error_code_returned:4", p.String())
b, err := protojson.Marshal(p)
require.NoError(t, err)
// Sometimes
// * "{\"value\":\"something\",\"errorCodeReturned\":4}
// * "{\"value\":\"something\", \"errorCodeReturned\":4}
assert.Equal(t, "{\"value\":\"something\",\"errorCodeReturned\":4}", string(b))
}
This passes for me 100%.
/home/bwplotka/.gvm/gos/go1.15/bin/go tool test2json -t /tmp/___TestNonDeterministicMarshal_in_github_com_grpc_ecosystem_go_grpc_middleware_v2_grpctesting_testpb -test.v -test.run ^\QTestNonDeterministicMarshal\E$
=== RUN TestNonDeterministicMarshal
--- PASS: TestNonDeterministicMarshal (0.00s)
PASS
Process finished with exit code 0
- Remove comment
// Remove this line to see the test failing (:
and newline below:
package testpb
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/encoding/protojson"
)
func TestNonDeterministicMarshal(t *testing.T) {
p := &PingRequest{Value: "something", ErrorCodeReturned: 4}
// Sometimes
// * "value:\"something\" error_code_returned:4"
// * "value:\"something\" error_code_returned:4"
assert.Equal(t, "value:\"something\" error_code_returned:4", p.String())
b, err := protojson.Marshal(p)
require.NoError(t, err)
// Sometimes
// * "{\"value\":\"something\",\"errorCodeReturned\":4}
// * "{\"value\":\"something\", \"errorCodeReturned\":4}
assert.Equal(t, "{\"value\":\"something\",\"errorCodeReturned\":4}", string(b))
}
No this fails for me with:
//home/bwplotka/.gvm/gos/go1.15/bin/go tool test2json -t /tmp/___TestNonDeterministicMarshal_in_github_com_grpc_ecosystem_go_grpc_middleware_v2_grpctesting_testpb -test.v -test.run ^\QTestNonDeterministicMarshal\E$
=== RUN TestNonDeterministicMarshal
test_repro_test.go:17:
Error Trace: test_repro_test.go:17
Error: Not equal:
expected: "value:\"something\" error_code_returned:4"
actual : "value:\"something\" error_code_returned:4"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-value:"something" error_code_returned:4
+value:"something" error_code_returned:4
Test: TestNonDeterministicMarshal
test_repro_test.go:24:
Error Trace: test_repro_test.go:24
Error: Not equal:
expected: "{\"value\":\"something\",\"errorCodeReturned\":4}"
actual : "{\"value\":\"something\", \"errorCodeReturned\":4}"
Diff:
--- Expected
+++ Actual
@@ -1 +1 @@
-{"value":"something","errorCodeReturned":4}
+{"value":"something", "errorCodeReturned":4}
Test: TestNonDeterministicMarshal
--- FAIL: TestNonDeterministicMarshal (0.00s)
- Put any comment back in that spot and all works again.
NOTE: This works somehow underministically, so CI, running it with other tests vs single changes behavior (essentially on what line to put comment to affect output)
What did you expect to see?
No matter what and where I put comments in my code I expect the same String
and protojson
Output =D
What did you see instead?
Putting comment in right spot adds or removes whitespace between poto fields.
Anything else we should know about your project / environment?
Pretty standard, you can see all here: https://github.com/grpc-ecosystem/go-grpc-middleware/tree/repro
We will not touch repro
branch so you can take a look.
I know it might be annoying little thing, but deterministic output would be supper nice, especially for reliable unit tests (that's how I noticed this)
created time in 3 days
issue commentgolang/protobuf
protoc-gen-go-grpc: program not found or is not executable when using APIv2
SOLVED: updated path to:
export GOPATH=$HOME/Documents/goworkspace export PATH="$PATH:$GOPATH/bin"
thank you
comment created time in 3 days
issue commentgolang/protobuf
protoc-gen-go-grpc: program not found or is not executable when using APIv2
@dbyington thank you
comment created time in 3 days
issue openedgogo/protobuf
EnumsAsInts does not work in jsonpb config
The code is so simple:
package main
import (
"os"
"github.com/gogo/protobuf/jsonpb"
"github.com/gogo/protobuf/proto"
)
func main() {
marshaller := jsonpb.Marshaler{
EnumsAsInts: true,
EmitDefaults: false,
OrigName: true,
}
marshaller.Marshal(os.Stdout, &TestInt{
Ori: 16,
})
}
type TestInt struct {
Ori uint64 `protobuf:"varint,1,opt,name=ori,proto3" json:"ori,omitempty"`
}
func (m *TestInt) Reset() { *m = TestInt{} }
func (m *TestInt) String() string { return proto.CompactTextString(m) }
func (*TestInt) ProtoMessage() {}
No matter I set EnumsAsInts: true
or not, all that I got is {"ori":"16"}
.
But what I want is {"ori":16}
created time in 4 days
PR closed gogo/protobuf
See https://github.com/golang/go/issues/32479
Signed-off-by: Robert-André Mauchin zebob.m@gmail.com
pr closed time in 4 days
issue commentgolang/protobuf
proto: repeated imported enum cause panic
If you can give us actual code with a reproduction we can look into it, but without a way to reproduce the problem I'm afraid there's not much we can do.
comment created time in 4 days
issue commentgolang/protobuf
proto: repeated imported enum cause panic
I’ve setup a gRPC service with the given proto files, and… it doesn’t crash. It just works, as expected.
comment created time in 4 days
startedwajahatkarim3/hugo-on-fire
started time in 4 days
issue commentgolang/protobuf
proto: repeated imported enum cause panic
https://github.com/mesg-foundation/engine/issues/1805 also reports the same bug after upgrading to v1.4+
comment created time in 4 days
issue commentgolang/protobuf
proto: repeated imported enum cause panic
@dsnet We also just ran into this, so I think the issue should be reopened. I'll try to summarise the problem below.
If you have two proto files:
// services.proto
import "common.proto";
message ListServicesRequest {
repeated Status filter_statuses = 1;
}
// common.proto
enum Status {
NOT_SET = 0;
DISABLED = 1;
ENABLED = 2;
}
whenever you call a RPC method that uses ListServicesRequest
, the following error is returned:
2021/01/15 12:19:32 http: panic serving 10.125.39.64:32176: runtime error: index out of range [0] with length 0
goroutine 159 [running]:
net/http.(*conn).serve.func1(0xc00056e640)
/usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xfdeac0, 0xc0004b6a60)
/usr/local/go/src/runtime/panic.go:975 +0x3e9
google.golang.org/protobuf/internal/filedesc.(*EnumValues).Get(0xc0000eaa80, 0x0, 0xc0000eaa80, 0xc0003e19a0)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/filedesc/desc_list_gen.go:66 +0x4c
google.golang.org/protobuf/internal/impl.newEnumConverter(0x11d5240, 0xef5060, 0x11d5120, 0xc0002be380, 0xc00000e758, 0xd68861a761eb97d8)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert.go:379 +0x8b
google.golang.org/protobuf/internal/impl.newSingularConverter(0x11d5240, 0xef5060, 0x11d5120, 0xc0002be380, 0xc0003e19e8, 0x8de86b)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert.go:138 +0xdfb
google.golang.org/protobuf/internal/impl.newListConverter(0x11d5240, 0xc00043a680, 0x11d5120, 0xc0002be380, 0xe6b320, 0x0)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert_list.go:17 +0x1f4
google.golang.org/protobuf/internal/impl.NewConverter(0x11d5240, 0xc00043a680, 0x11d5120, 0xc0002be380, 0x11ccc00, 0xc0004076b0)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert.go:56 +0x126
google.golang.org/protobuf/internal/impl.fieldInfoForList(0x11d5120, 0xc0002be380, 0xfa7953, 0xe, 0x0, 0x0, 0x11d5240, 0xe74d80, 0xfa7963, 0xa0, ...)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect_field.go:172 +0xe5
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeKnownFieldsFunc(0xc0005f4140, 0x58, 0xffffffffffffffff, 0x40, 0xffffffffffffffff, 0xc0004072f0, 0xc000407320, 0xc000407350, 0xc000407380)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect.go:61 +0x69d
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeReflectFuncs(0xc0005f4140, 0x11d5240, 0x1004600, 0x58, 0xffffffffffffffff, 0x40, 0xffffffffffffffff, 0xc0004072f0, 0xc000407320, 0xc000407350, ...)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect.go:36 +0x65
google.golang.org/protobuf/internal/impl.(*MessageInfo).initOnce(0xc0005f4140)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message.go:90 +0x192
google.golang.org/protobuf/internal/impl.(*MessageInfo).init(...)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message.go:72
google.golang.org/protobuf/internal/impl.(*messageReflectWrapper).ProtoMethods(0xc0004b4a10, 0xc0003e2158)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect_gen.go:150 +0x53
google.golang.org/protobuf/proto.protoMethods(...)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/proto/proto_methods.go:18
google.golang.org/protobuf/proto.MarshalOptions.marshal(0xc000000001, 0x0, 0x0, 0x0, 0x11d2de0, 0xc0004b4a10, 0xc0004b4a10, 0x11d2de0, 0xc0004b4a10, 0x1027a20, ...)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/proto/encode.go:143 +0x49
google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend(0x1000001, 0x0, 0x0, 0x0, 0x11afde0, 0xc0004b4a10, 0x203000, 0x7f0c1b3effff, 0xc00, 0x8, ...)
/go/pkg/mod/google.golang.org/protobuf@v1.25.0/proto/encode.go:125 +0x98
github.com/golang/protobuf/proto.marshalAppend(0x0, 0x0, 0x0, 0x11c38e0, 0xc00011a5a0, 0xc00043a500, 0x40, 0x40, 0xfd1700, 0x418745, ...)
/go/pkg/mod/github.com/golang/protobuf@v1.4.2/proto/wire.go:40 +0xc7
github.com/golang/protobuf/proto.Marshal(...)
/go/pkg/mod/github.com/golang/protobuf@v1.4.2/proto/wire.go:23
This only occurs if the RPC method uses a request struct, which contains a repeated
field that references an Enum in a different file. So if your proto file imports that enum, it will cause an error.
Downgrading to v1.3.5
fixed this, so I think it's a regression from v1.4
onwards.
comment created time in 4 days
issue closedgolang/protobuf
How to stop protoc-gen-go from downgrading to the previous version (1.23.0)
I can't work out why all of a sudden when I run protoc -I . <service>.proto
it generates a pb.go file with an older version of protoc-gen-go (see image below)
Environment
- I have protobuf version 3.14.0 installed globally via homebrew (
brew info protobuf
orprotoc --version
) - in the project, I have
google.golang.org/protobuf v1.25.0
andgithub.com/golang/protobuf v1.4.3
installed via go.mod – but this happens for me across all the services which suggests the problem is to do with the global module - My go version is:
go1.15.6 darwin/amd64
(the latest)
I appreciate any suggestions. Thanks
closed time in 4 days
Neats29issue commentgolang/protobuf
How to stop protoc-gen-go from downgrading to the previous version (1.23.0)
Thank you. I resolved my issue by using a different version of protobuf.
comment created time in 4 days
issue commentgolang/protobuf
protoc-gen-go: packages named 'type' induce a malformed warnings
I later realized that, although this is annoying, it's not a blocking. I can still import the generated package via path and just ignore the package name.
comment created time in 6 days
startedmathworks/thingspeak-arduino
started time in 6 days
issue commentgolang/protobuf
reflect/protoregistry: conflicts with same filename
Has the protobuf team made any further decisions on this issue?
Much like @rcgoodfellow, I have a Go program that depends on two third-party libraries, each of which defines a telemetry.proto
file at the top level of their project. With current google.golang.org/protobuf, the program panics unless I set the GOLANG_PROTOBUF_REGISTRATION_CONFLICT environment variable.
This can't be that uncommon a situation. At this point, I'm not really clear how the protobuf team expect the file-by-path registry to work outside of a monorepo setting.
comment created time in 6 days
startedgookit/color
started time in 6 days
startedjba/templatecheck
started time in 7 days
issue commentgogo/protobuf
Starting on apiv2 - custom types
I think the basic idea would be to generate code for 'custom types' that implements protoreflect.Message
(actually protoreflect.ProtoMessage
but this just provides a layer of indirection between the custom type and the protobuf reflective type): https://godoc.org/google.golang.org/protobuf/reflect/protoreflect#Message.
This would be analogous the the various marshalling methods requried by gogoproto.customtype
: https://github.com/gogo/protobuf/blob/master/custom_types.md#custom-type-method-signatures.
Types that implement ProtoMessage
should use the Message
interface to reflect on the type. At that point it is possible to handle the mapping between the protobuf type system and an arbitrary Go type.
I have been exploring the API myself to see whether there are some types that are open for extension but with which I could share most of the implementation. Most of the Message
implementing types are internal
, here is an example: https://github.com/protocolbuffers/protobuf-go/blob/master/testing/protocmp/reflect.go#L39
comment created time in 7 days
fork aclements/klog
Leveled execution logs for Go (fork of https://github.com/golang/glog)
fork in 8 days
issue commentgolang/protobuf
How to stop protoc-gen-go from downgrading to the previous version (1.23.0)
I have google.golang.org/protobuf v1.25.0
installed via go.mod …
The protoc-gen-go
binary has to be v1.25.0, and it will not automatically use the value in the go.mod, you have to manage the binary yourself.
comment created time in 8 days
issue openedgolang/protobuf
How to stop `protoc-gen-go` downgrading to the previous version (1.23.0)
I can't work out why all of a sudden when I run protoc -I . <service>.proto
it generates a pb.go file with an older version of protoc-gen-go (see image below)
Environment
- I have protobuf version 3.14.0 installed globally via homebrew (
brew info protobuf
orprotoc --version
) - in the project, I have
google.golang.org/protobuf v1.25.0
andgithub.com/golang/protobuf v1.4.3
installed via go.mod - My go version is:
go1.15.6 darwin/amd64
(the latest)
I appreciate any suggestions. Thanks
created time in 8 days