In some scenarios, we need to render number64 type such as int64, uint64 as integers instead of string values, but found we cannot. So I add an option in config to fix this.

test/api/test.proto

syntax="proto3";
package test;
option go_package="test/api;api";
import "test/first/first.proto";
service test{
	rpc hello(hreq)returns(hresp);
}
message hreq{
	map<int32,first.a.abc> test=1;
}
message hresp{
	string w=1;
}

test/first/first.proto

syntax="proto3";
package first;
option go_package="test/first;first";
message a{
	enum abc{
		A = 0;
		B = 1;
		C = 2;
	}
}

when generate the test/api/test.proto how to get the first.a.abc go pkg and go name the first.a.abccan change to a message type,how to get the go package and go name

test/api/test.proto

syntax="proto3";
package test;
option go_package="test/api;api";
import "test/first/first.proto";
service test{
	rpc hello(hreq)returns(hresp);
}
message hreq{
	map<int32,first.a.abc> test=1;
}
message hresp{
	string w=1;
}

test/first/first.proto

syntax="proto3";
package first;
option go_package="test/first;first";
message a{
	enum abc{
		A = 0;
		B = 1;
		C = 2;
	}
}

when generate the test/api/test.proto how to get the first.a.abc go pkg and go name

Thanks a lot, sorry for duplicate!

What version of protobuf and what language are you using?

My go.mod:

	github.com/golang/protobuf v1.4.3
	github.com/opentracing/opentracing-go v1.1.0
	github.com/pkg/errors v0.9.1
	github.com/stretchr/testify v1.6.1
	golang.org/x/net v0.0.0-20190620200207-3b0461eec859
	golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
	google.golang.org/grpc v1.30.1
	google.golang.org/grpc/examples v0.0.0-20200723182653-9106c3fff523
	google.golang.org/protobuf v1.25.0

protoc-gen-go: require google.golang.org/protobuf v1.25.0 // cmd/protoc-gen-go (We use bingo for tool pinning)

Go 1.15

What did you do?

Okay, it's extremely weird so stay with me. (:

The repro test is here https://github.com/grpc-ecosystem/go-grpc-middleware/blob/repro/grpctesting/testpb/test_repro_test.go you can find proto as well.

Now it's tricky to reproduce, because it depends on... code line you invoke String or protojson? Like some compiler optimization is changing things? I know it's super weird, but it looks like this:

Steps to repro (I am using Goland test invocation):

1. Run test.

package testpb

import (
	"testing"

	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
	"google.golang.org/protobuf/encoding/protojson"
)

func TestNonDeterministicMarshal(t *testing.T) {
	p := &PingRequest{Value: "something", ErrorCodeReturned: 4}

	// Remove this line to see the test failing (:

	// Sometimes
	// * "value:\"something\" error_code_returned:4"
	// * "value:\"something\"  error_code_returned:4"
	assert.Equal(t, "value:\"something\" error_code_returned:4", p.String())
	b, err := protojson.Marshal(p)
	require.NoError(t, err)

	// Sometimes
	// * "{\"value\":\"something\",\"errorCodeReturned\":4}
	// * "{\"value\":\"something\", \"errorCodeReturned\":4}
	assert.Equal(t, "{\"value\":\"something\",\"errorCodeReturned\":4}", string(b))
}

This passes for me 100%.

/home/bwplotka/.gvm/gos/go1.15/bin/go tool test2json -t /tmp/___TestNonDeterministicMarshal_in_github_com_grpc_ecosystem_go_grpc_middleware_v2_grpctesting_testpb -test.v -test.run ^\QTestNonDeterministicMarshal\E$
=== RUN   TestNonDeterministicMarshal
--- PASS: TestNonDeterministicMarshal (0.00s)
PASS

Process finished with exit code 0

2. Remove comment // Remove this line to see the test failing (: and newline below:

package testpb

import (
	"testing"

	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
	"google.golang.org/protobuf/encoding/protojson"
)

func TestNonDeterministicMarshal(t *testing.T) {
	p := &PingRequest{Value: "something", ErrorCodeReturned: 4}

	// Sometimes
	// * "value:\"something\" error_code_returned:4"
	// * "value:\"something\"  error_code_returned:4"
	assert.Equal(t, "value:\"something\" error_code_returned:4", p.String())
	b, err := protojson.Marshal(p)
	require.NoError(t, err)

	// Sometimes
	// * "{\"value\":\"something\",\"errorCodeReturned\":4}
	// * "{\"value\":\"something\", \"errorCodeReturned\":4}
	assert.Equal(t, "{\"value\":\"something\",\"errorCodeReturned\":4}", string(b))
}

No this fails for me with:

//home/bwplotka/.gvm/gos/go1.15/bin/go tool test2json -t /tmp/___TestNonDeterministicMarshal_in_github_com_grpc_ecosystem_go_grpc_middleware_v2_grpctesting_testpb -test.v -test.run ^\QTestNonDeterministicMarshal\E$
=== RUN   TestNonDeterministicMarshal
    test_repro_test.go:17: 
        	Error Trace:	test_repro_test.go:17
        	Error:      	Not equal: 
        	            	expected: "value:\"something\" error_code_returned:4"
        	            	actual  : "value:\"something\"  error_code_returned:4"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-value:"something" error_code_returned:4
        	            	+value:"something"  error_code_returned:4
        	Test:       	TestNonDeterministicMarshal
    test_repro_test.go:24: 
        	Error Trace:	test_repro_test.go:24
        	Error:      	Not equal: 
        	            	expected: "{\"value\":\"something\",\"errorCodeReturned\":4}"
        	            	actual  : "{\"value\":\"something\", \"errorCodeReturned\":4}"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-{"value":"something","errorCodeReturned":4}
        	            	+{"value":"something", "errorCodeReturned":4}
        	Test:       	TestNonDeterministicMarshal
--- FAIL: TestNonDeterministicMarshal (0.00s)

3. Put any comment back in that spot and all works again.

NOTE: This works somehow underministically, so CI, running it with other tests vs single changes behavior (essentially on what line to put comment to affect output)

What did you expect to see?

No matter what and where I put comments in my code I expect the same String and protojson Output =D

What did you see instead?

Putting comment in right spot adds or removes whitespace between poto fields.

Anything else we should know about your project / environment?

Pretty standard, you can see all here: https://github.com/grpc-ecosystem/go-grpc-middleware/tree/repro

We will not touch repro branch so you can take a look.

I know it might be annoying little thing, but deterministic output would be supper nice, especially for reliable unit tests (that's how I noticed this)

SOLVED: updated path to:

export GOPATH=$HOME/Documents/goworkspace
export PATH="$PATH:$GOPATH/bin"

thank you

@dbyington thank you

The code is so simple:

package main

import (
	"os"

	"github.com/gogo/protobuf/jsonpb"
	"github.com/gogo/protobuf/proto"
)

func main() {
	marshaller := jsonpb.Marshaler{
		EnumsAsInts:  true,
		EmitDefaults: false,
		OrigName:     true,
	}

	marshaller.Marshal(os.Stdout, &TestInt{
		Ori: 16,
	})
}

type TestInt struct {
	Ori uint64 `protobuf:"varint,1,opt,name=ori,proto3" json:"ori,omitempty"`
}

func (m *TestInt) Reset()         { *m = TestInt{} }
func (m *TestInt) String() string { return proto.CompactTextString(m) }
func (*TestInt) ProtoMessage()    {}

No matter I set EnumsAsInts: true or not, all that I got is {"ori":"16"}. But what I want is {"ori":16}

See https://github.com/golang/go/issues/32479

Signed-off-by: Robert-André Mauchin zebob.m@gmail.com

If you can give us actual code with a reproduction we can look into it, but without a way to reproduce the problem I'm afraid there's not much we can do.

I’ve setup a gRPC service with the given proto files, and… it doesn’t crash. It just works, as expected.

https://github.com/mesg-foundation/engine/issues/1805 also reports the same bug after upgrading to v1.4+

@dsnet We also just ran into this, so I think the issue should be reopened. I'll try to summarise the problem below.

If you have two proto files:

// services.proto

import "common.proto";

message ListServicesRequest {
    repeated Status filter_statuses = 1;
}

// common.proto

enum Status {
    NOT_SET = 0;
    DISABLED = 1;
    ENABLED = 2;
}

whenever you call a RPC method that uses ListServicesRequest, the following error is returned:

2021/01/15 12:19:32 http: panic serving 10.125.39.64:32176: runtime error: index out of range [0] with length 0
goroutine 159 [running]:
net/http.(*conn).serve.func1(0xc00056e640)
        /usr/local/go/src/net/http/server.go:1801 +0x147
panic(0xfdeac0, 0xc0004b6a60)
        /usr/local/go/src/runtime/panic.go:975 +0x3e9
google.golang.org/protobuf/internal/filedesc.(*EnumValues).Get(0xc0000eaa80, 0x0, 0xc0000eaa80, 0xc0003e19a0)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/filedesc/desc_list_gen.go:66 +0x4c
google.golang.org/protobuf/internal/impl.newEnumConverter(0x11d5240, 0xef5060, 0x11d5120, 0xc0002be380, 0xc00000e758, 0xd68861a761eb97d8)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert.go:379 +0x8b
google.golang.org/protobuf/internal/impl.newSingularConverter(0x11d5240, 0xef5060, 0x11d5120, 0xc0002be380, 0xc0003e19e8, 0x8de86b)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert.go:138 +0xdfb
google.golang.org/protobuf/internal/impl.newListConverter(0x11d5240, 0xc00043a680, 0x11d5120, 0xc0002be380, 0xe6b320, 0x0)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert_list.go:17 +0x1f4
google.golang.org/protobuf/internal/impl.NewConverter(0x11d5240, 0xc00043a680, 0x11d5120, 0xc0002be380, 0x11ccc00, 0xc0004076b0)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/convert.go:56 +0x126
google.golang.org/protobuf/internal/impl.fieldInfoForList(0x11d5120, 0xc0002be380, 0xfa7953, 0xe, 0x0, 0x0, 0x11d5240, 0xe74d80, 0xfa7963, 0xa0, ...)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect_field.go:172 +0xe5
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeKnownFieldsFunc(0xc0005f4140, 0x58, 0xffffffffffffffff, 0x40, 0xffffffffffffffff, 0xc0004072f0, 0xc000407320, 0xc000407350, 0xc000407380)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect.go:61 +0x69d
google.golang.org/protobuf/internal/impl.(*MessageInfo).makeReflectFuncs(0xc0005f4140, 0x11d5240, 0x1004600, 0x58, 0xffffffffffffffff, 0x40, 0xffffffffffffffff, 0xc0004072f0, 0xc000407320, 0xc000407350, ...)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect.go:36 +0x65
google.golang.org/protobuf/internal/impl.(*MessageInfo).initOnce(0xc0005f4140)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message.go:90 +0x192
google.golang.org/protobuf/internal/impl.(*MessageInfo).init(...)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message.go:72
google.golang.org/protobuf/internal/impl.(*messageReflectWrapper).ProtoMethods(0xc0004b4a10, 0xc0003e2158)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/internal/impl/message_reflect_gen.go:150 +0x53
google.golang.org/protobuf/proto.protoMethods(...)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/proto/proto_methods.go:18
google.golang.org/protobuf/proto.MarshalOptions.marshal(0xc000000001, 0x0, 0x0, 0x0, 0x11d2de0, 0xc0004b4a10, 0xc0004b4a10, 0x11d2de0, 0xc0004b4a10, 0x1027a20, ...)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/proto/encode.go:143 +0x49
google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend(0x1000001, 0x0, 0x0, 0x0, 0x11afde0, 0xc0004b4a10, 0x203000, 0x7f0c1b3effff, 0xc00, 0x8, ...)
        /go/pkg/mod/google.golang.org/protobuf@v1.25.0/proto/encode.go:125 +0x98
github.com/golang/protobuf/proto.marshalAppend(0x0, 0x0, 0x0, 0x11c38e0, 0xc00011a5a0, 0xc00043a500, 0x40, 0x40, 0xfd1700, 0x418745, ...)
        /go/pkg/mod/github.com/golang/protobuf@v1.4.2/proto/wire.go:40 +0xc7
github.com/golang/protobuf/proto.Marshal(...)
        /go/pkg/mod/github.com/golang/protobuf@v1.4.2/proto/wire.go:23

This only occurs if the RPC method uses a request struct, which contains a repeated field that references an Enum in a different file. So if your proto file imports that enum, it will cause an error.

Downgrading to v1.3.5 fixed this, so I think it's a regression from v1.4 onwards.

I can't work out why all of a sudden when I run protoc -I . <service>.proto it generates a pb.go file with an older version of protoc-gen-go (see image below)

Environment

• I have protobuf version 3.14.0 installed globally via homebrew (brew info protobuf or protoc --version)
• in the project, I have google.golang.org/protobuf v1.25.0 and github.com/golang/protobuf v1.4.3 installed via go.mod – but this happens for me across all the services which suggests the problem is to do with the global module
• My go version is: go1.15.6 darwin/amd64 (the latest)

I appreciate any suggestions. Thanks

Thank you. I resolved my issue by using a different version of protobuf.

I later realized that, although this is annoying, it's not a blocking. I can still import the generated package via path and just ignore the package name.

Has the protobuf team made any further decisions on this issue?

Much like @rcgoodfellow, I have a Go program that depends on two third-party libraries, each of which defines a telemetry.proto file at the top level of their project. With current google.golang.org/protobuf, the program panics unless I set the GOLANG_PROTOBUF_REGISTRATION_CONFLICT environment variable.

This can't be that uncommon a situation. At this point, I'm not really clear how the protobuf team expect the file-by-path registry to work outside of a monorepo setting.

I think the basic idea would be to generate code for 'custom types' that implements protoreflect.Message (actually protoreflect.ProtoMessage but this just provides a layer of indirection between the custom type and the protobuf reflective type): https://godoc.org/google.golang.org/protobuf/reflect/protoreflect#Message.

This would be analogous the the various marshalling methods requried by gogoproto.customtype: https://github.com/gogo/protobuf/blob/master/custom_types.md#custom-type-method-signatures.

Types that implement ProtoMessage should use the Message interface to reflect on the type. At that point it is possible to handle the mapping between the protobuf type system and an arbitrary Go type.

I have been exploring the API myself to see whether there are some types that are open for extension but with which I could share most of the implementation. Most of the Message implementing types are internal, here is an example: https://github.com/protocolbuffers/protobuf-go/blob/master/testing/protocmp/reflect.go#L39

I have google.golang.org/protobuf v1.25.0 installed via go.mod …

The protoc-gen-go binary has to be v1.25.0, and it will not automatically use the value in the go.mod, you have to manage the binary yourself.

I can't work out why all of a sudden when I run protoc -I . <service>.proto it generates a pb.go file with an older version of protoc-gen-go (see image below)

Environment

• I have protobuf version 3.14.0 installed globally via homebrew (brew info protobuf or protoc --version)
• in the project, I have google.golang.org/protobuf v1.25.0 and github.com/golang/protobuf v1.4.3 installed via go.mod
• My go version is: go1.15.6 darwin/amd64 (the latest)

I appreciate any suggestions. Thanks