profile
viewpoint
Elliotte Rusty Harold elharo New York https://www.elharo.com/blog/ Elliotte Rusty Harold lives in a secret mountaintop laboratory on a large island off the East Coast of the United States. He's an active member of SFWA and BSFW

elharo/aether-demo-maven-plugin 0

A simple Maven plugin demonstrating the use of Aether.

elharo/appengine-maven-archetypes-java 0

App Engine Maven Archetypes

elharo/appengine-php-wordpress-starter-project 0

Starter project for running WordPress on Google Cloud Platform

elharo/appengine-plugins-core 0

A client Java library to manage App Engine Java applications for any project that performs App Engine Java application management. For example, the Maven, Gradle and Eclipse App Engine plugins, custom user tools, etc.

elharo/auto 0

A collection of source code generators for Java.

elharo/avro 0

Apache Avro

elharo/bazel 0

a fast, scalable, multi-language and extensible build system

issue commentprotocolbuffers/protobuf

Unexpected Behavior from EnumValueToPascalCase

Would any such change be backwards compatible?

mchristen-astranis

comment created time in a day

push eventelharo/protobuf

Elliotte Rusty Harold

commit sha 0af343daf5add02a5ff7451c5b04bc70794ca98f

remove from Makefile

view details

push time in 2 days

PR opened protocolbuffers/protobuf

delete unused test proto java release notes: no
+0 -44

0 comment

1 changed file

pr created time in 2 days

create barnchelharo/protobuf

branch : proto

created branch time in 2 days

push eventelharo/protobuf

James Newton-King

commit sha 22462b0c529b444d065679885a6aaf5cb2514e05

Fix trim warnings

view details

Elliotte Rusty Harold

commit sha 278dfb24964ff12702dc21dc9d49e20f0a4345b4

remove UTF8Utils class that hasn't been needed since Java 6

view details

Elliotte Rusty Harold

commit sha 473ff00de394b55adf96ad18285a42e2dc2d5479

Merge pull request #9266 from elharo/utf8utils remove UTF8Utils class that hasn't been needed since Java 6

view details

Deanna Garcia

commit sha 667d5e9e9f1a9ebc15cfcddb5f272a301d0f16b6

No error throwing for unknown OS

view details

deannagarcia

commit sha d69e482e5aebb0daea7bbdf54232332116b1fd6f

Merge pull request #9267 from protocolbuffers/protocNaming No error throwing for unknown OS

view details

Jan Tattermusch

commit sha b79ac0e6e294019bf010cbdad2fd3f3af07ed423

Merge pull request #9182 from JamesNK/jamesnk/trimming-take3 [C#] Fix trim warnings

view details

Adam Cozzette

commit sha 9a1d5dfa4bfcddac7e7db71c899aa0b832161bcc

Small tweaks to C# unit tests (#9271) This change includes just a couple small tweaks: - Keep line lengths under 80 characters - Fully qualify google::protobuf::Any so that the code still compiles internally where we use a different namespace.

view details

push time in 2 days

issue commentprotocolbuffers/protobuf

Define type-safe enums in Ruby generated code using Sorbet

I'm not familiar with Ruby. Is Sorbet an extra dependency? Would this be fully backwards and forwards compatible with existing code?

chnakamura

comment created time in 3 days

issue commentprotocolbuffers/protobuf

the result of serializing same data is not same in Javascript and golang

OK, that makes a little more sense. That said I think there's something inobvious happening here that has nothing to do with protos. What happens when you go JavaScript to JavaScript? JavaScript (not protos but JavaScript itself) represents all numbers as IEEE 754 doubles which do not have as much precision in the integer part as a true int64 or long. I suspect your long value is rounded because JavaScript cannot precisely represent 1638243612614987000.

See https://stackoverflow.com/questions/5353388/javascript-parsing-int64

kysee

comment created time in 4 days

issue commentprotocolbuffers/protobuf

Support data redaction, so that sensitive data in protos doesn't leak into places like log files

This feels similar in spirit to https://github.com/protocolbuffers/protobuf/issues/9114 though the details are quite different.

Bottom line: the protobuf libraries do not support custom input or output formats, nor is this a goal. The proto libraries are designed to consume and produce very specific, documented formats as used by gRPC and many other systems. Any other format — protobuf, JSON, or something else — can be created or consumed by writing code to filter the standard inputs and output as needed by a specific application. Indeed this is such a common thing to do that it's become a Meme about SWE careers at Google. However this code is in a separate layer and does not need to be committed to the core libraries in this repo.

jhump

comment created time in 4 days

issue closedprotocolbuffers/protobuf

the result of serializing same data is not same in Javascript and golang

<!--

NOTE: this form is for bug reports only. For questions or troubleshooting, please post on the protobuf mailing list: https://groups.google.com/forum/#!forum/protobuf Stack Overflow is also a useful if unofficial resource https://stackoverflow.com/questions/tagged/protocol-buffers -->

What version of protobuf and what language are you using? Version:

$ protoc --version
libprotoc 3.17.3
  • javascrip(node module) "google-protobuf": "^3.18.0-rc.2",
  • golang pakcate1: google.golang.org/protobuf v1.27.1
  • golang pakcate2: github.com/gogo/protobuf v1.3.2

Language: Javascript and Golang

What operating system (Linux, Windows, ...) and version? OSX Monterey 12.0.1

What runtime / compiler are you using (e.g., python version or gcc version)

  • Javascript environment
    • node v14.17.6
    • npm module "google-protobuf": "^3.18.0-rc.2",
  • Golang environment
    • go1.16.2 darwin/amd64
    • google.golang.org/protobuf v1.27.1
    • github.com/gogo/protobuf v1.3.2

What did you do? Steps to reproduce the behavior:

  1. Write prototest.proto for a proto3 message including int64 type.
syntax = "proto3";
package tmp
option go_package = "my_project_path/tmp";

message Int64Msg {
  int64 i64 = 1;
}
  1. Generate go source code protoc --go_out=$(GOPATH)/src -I./proto/ prototest.proto

  2. Run the following code in go.

func TestInt64(t *testing.T) {
	st := &Int64Msg{}
	st.I64 = 1638243612614987000

	bz, err := proto.Marshal(st)
	require.NoError(t, err)
	fmt.Printf("%x\n", bz0))
}

Output is 08f899e0cbacbc8dde16

  1. Generate javascript source code protoc --js_out=import_style=commonjs,binary:./proto -I./proto/ prototest.proto

  2. Run the following code in javascript.

import Proto from '../proto/prototest_pb.cjs';

let st = new Proto.Int64Msg();
st.setI64(1638243612614987000);
let bz = st.serializeBinary();
console.debug(Buffer.from(bz).toString('hex'));

Output is 08809ae0cbacbc8dde16

  1. golang's output (08f899e0cbacbc8dde16) is not equal to Javascript's output(08809ae0cbacbc8dde16)

What did you expect to see

The result of serializing the same data should be the same in both javascript and golang.

What did you see instead?

As mentioned,
golang's output (08f899e0cbacbc8dde16) is not equal to Javascript's output(08809ae0cbacbc8dde16)

Anything else we should know about your project / environment

closed time in 4 days

kysee

issue commentprotocolbuffers/protobuf

the result of serializing same data is not same in Javascript and golang

Generally proto serialization is not deterministic at the byte level, not even within the same language binding, much less between languages. As long as the protos decode the same given the same schema, there's no bug here.

kysee

comment created time in 4 days

issue closedprotocolbuffers/protobuf

objc UTF-8 failure, is some field type 'string' when it should be 'bytes'?

<!--

NOTE: this form is for bug reports only. For questions or troubleshooting, please post on the protobuf mailing list: https://groups.google.com/forum/#!forum/protobuf Stack Overflow is also a useful if unofficial resource https://stackoverflow.com/questions/tagged/protocol-buffers -->

What version of protobuf and what language are you using? Version: master/v3.6.0/v3.5.0 etc. Language: Objective-C

After objectivec integration, parse the pb data returned by Java to prompt the problem ‘UTF-8 failure, is some field type 'string' when it should be 'bytes'?’ how to do?

closed time in 4 days

XinlinLiu

issue commentprotocolbuffers/protobuf

objc UTF-8 failure, is some field type 'string' when it should be 'bytes'?

For questions or troubleshooting, please see the protobuf mailing list: https://groups.google.com/forum/#!forum/protobuf or Stack Overflow https://stackoverflow.com/questions/tagged/protocol-buffers

We don't formally monitor stack overflow, but there are a lot of experienced developers there who also use protobuf.

XinlinLiu

comment created time in 4 days

issue commentprotocolbuffers/protobuf

stdout/stderr/stdin are allowed fields in C++ despite being reserved by the C specification

Do you have a test case? Might be just my limited C++ knowledge here, but it's not obvious to me what exactly you're suggesting.

deeglaze

comment created time in 5 days

push eventprotocolbuffers/protobuf

Elliotte Rusty Harold

commit sha 278dfb24964ff12702dc21dc9d49e20f0a4345b4

remove UTF8Utils class that hasn't been needed since Java 6

view details

Elliotte Rusty Harold

commit sha 473ff00de394b55adf96ad18285a42e2dc2d5479

Merge pull request #9266 from elharo/utf8utils remove UTF8Utils class that hasn't been needed since Java 6

view details

push time in 5 days

delete branch elharo/protobuf

delete branch : utf8utils

delete time in 5 days

PR merged protocolbuffers/protobuf

remove UTF8Utils class that hasn't been needed since Java 6 cla: yes java release notes: no

@acozzette

+1 -194

0 comment

3 changed files

elharo

pr closed time in 5 days

PR opened protocolbuffers/protobuf

Reviewers
remove UTF8Utils class that hasn't been needed since Java 6 java release notes: no

@acozzette

+1 -194

0 comment

3 changed files

pr created time in 5 days

create barnchelharo/protobuf

branch : utf8utils

created branch time in 5 days

push eventelharo/protobuf

Elliotte Rusty Harold

commit sha 19eaec02e1cdb2d8747c4d34fa0ee8f555d9d560

delete unused files (#9262) * delete unused files * update Makefile

view details

push time in 5 days

push eventprotocolbuffers/protobuf

Elliotte Rusty Harold

commit sha 19eaec02e1cdb2d8747c4d34fa0ee8f555d9d560

delete unused files (#9262) * delete unused files * update Makefile

view details

push time in 5 days

delete branch elharo/protobuf

delete branch : unused

delete time in 5 days

PR merged protocolbuffers/protobuf

delete unused files cla: yes java release notes: no

@acozzette

+0 -1400

0 comment

3 changed files

elharo

pr closed time in 5 days

PullRequestReviewEvent
PullRequestReviewEvent

pull request commentprotocolbuffers/protobuf

Add cmake option `protobuf_INSTALL` to not install files

Please resolve conflicts

jcooky

comment created time in 5 days

Pull request review commentprotocolbuffers/protobuf

Allow multiple input files for --dependency_out

 bool CommandLineInterface::GenerateDependencyManifestFile(   io::FileOutputStream out(fd);   io::Printer printer(&out, '$'); -  for (int i = 0; i < output_filenames.size(); i++) {-    printer.Print(output_filenames[i].c_str());-    if (i == output_filenames.size() - 1) {-      printer.Print(":");-    } else {-      printer.Print(" \\\n");-    }-  }+  auto ResolvePath = [&source_tree](const FileDescriptor* descriptor) {+        const std::string& virtual_file = descriptor->name();+        std::string disk_file;+        if (source_tree->VirtualFileToDiskFile(virtual_file, &disk_file)) {+          return disk_file;+        } else {+          std::cerr << "Unable to identify path for file " << virtual_file+                    << std::endl;+          return std::string();+        }+  }; -  for (int i = 0; i < file_set.file_size(); i++) {-    const FileDescriptorProto& file = file_set.file(i);-    const std::string& virtual_file = file.name();-    std::string disk_file;-    if (source_tree &&-        source_tree->VirtualFileToDiskFile(virtual_file, &disk_file)) {-      printer.Print(" $disk_file$", "disk_file", disk_file);-      if (i < file_set.file_size() - 1) printer.Print("\\\n");-    } else {-      std::cerr << "Unable to identify path for file " << virtual_file-                << std::endl;-      return false;+  for (const auto& pair : output_directories) {+    const std::string& location = pair.first;+    GeneratorContextImpl* directory = pair.second.get();+    std::map<const FileDescriptor *, std::vector<std::string>> descriptor_to_paths;+    directory->ListDescriptorToPathMappings(&descriptor_to_paths);+    for (auto it = descriptor_to_paths.rbegin(); it != descriptor_to_paths.rend(); it++)+    {+        const auto pair = *it;+        const FileDescriptor* descriptor = pair.first;+        const auto& paths = pair.second;+        for (size_t i = 0; i < paths.size(); i++)+        {+          std::string output_filename = location + paths[i];+          if (output_filename.compare(0, 2, "./") == 0) {+            output_filename = output_filename.substr(2);+          }+          printer.Print("$filename$$eol_escape$",+          "filename", output_filename.c_str(),+          "eol_escape", i != paths.size()- 1 ? " \\\n" : "");+        }+        std::string disk_file = ResolvePath(descriptor);+        if (disk_file.empty()) return false;+        printer.Print(": \\\n\t$disk_file$\n\n", "disk_file", disk_file);+    }+    for (auto it = already_seen.begin(); it != already_seen.end(); it++)

ditto

sweetgiorni

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentprotocolbuffers/protobuf

Allow multiple input files for --dependency_out

 bool CommandLineInterface::GenerateDependencyManifestFile(   io::FileOutputStream out(fd);   io::Printer printer(&out, '$'); -  for (int i = 0; i < output_filenames.size(); i++) {-    printer.Print(output_filenames[i].c_str());-    if (i == output_filenames.size() - 1) {-      printer.Print(":");-    } else {-      printer.Print(" \\\n");-    }-  }+  auto ResolvePath = [&source_tree](const FileDescriptor* descriptor) {+        const std::string& virtual_file = descriptor->name();+        std::string disk_file;+        if (source_tree->VirtualFileToDiskFile(virtual_file, &disk_file)) {+          return disk_file;+        } else {+          std::cerr << "Unable to identify path for file " << virtual_file+                    << std::endl;+          return std::string();+        }+  }; -  for (int i = 0; i < file_set.file_size(); i++) {-    const FileDescriptorProto& file = file_set.file(i);-    const std::string& virtual_file = file.name();-    std::string disk_file;-    if (source_tree &&-        source_tree->VirtualFileToDiskFile(virtual_file, &disk_file)) {-      printer.Print(" $disk_file$", "disk_file", disk_file);-      if (i < file_set.file_size() - 1) printer.Print("\\\n");-    } else {-      std::cerr << "Unable to identify path for file " << virtual_file-                << std::endl;-      return false;+  for (const auto& pair : output_directories) {+    const std::string& location = pair.first;+    GeneratorContextImpl* directory = pair.second.get();+    std::map<const FileDescriptor *, std::vector<std::string>> descriptor_to_paths;+    directory->ListDescriptorToPathMappings(&descriptor_to_paths);+    for (auto it = descriptor_to_paths.rbegin(); it != descriptor_to_paths.rend(); it++)+    {+        const auto pair = *it;+        const FileDescriptor* descriptor = pair.first;+        const auto& paths = pair.second;+        for (size_t i = 0; i < paths.size(); i++)+        {+          std::string output_filename = location + paths[i];+          if (output_filename.compare(0, 2, "./") == 0) {+            output_filename = output_filename.substr(2);+          }+          printer.Print("$filename$$eol_escape$",+          "filename", output_filename.c_str(),+          "eol_escape", i != paths.size()- 1 ? " \\\n" : "");+        }+        std::string disk_file = ResolvePath(descriptor);+        if (disk_file.empty()) return false;+        printer.Print(": \\\n\t$disk_file$\n\n", "disk_file", disk_file);+    }+    for (auto it = already_seen.begin(); it != already_seen.end(); it++)+    {+      const FileDescriptor* descriptor = *it;+      if (!descriptor->dependency_count()) continue;+      std::string disk_file = ResolvePath(descriptor);+      if (disk_file.empty()) return false;+      printer.Print("$name$: \\\n", "name", disk_file);+      for (int i = 0; i < descriptor->dependency_count(); i++)+      {

ditto

sweetgiorni

comment created time in 5 days

PullRequestReviewEvent

Pull request review commentprotocolbuffers/protobuf

[cleanup] Msvc warnings

 void IoWin32Test::SetUp() {   test_tmpdir.clear();   wtest_tmpdir.clear();   DWORD size = ::GetCurrentDirectoryW(MAX_PATH, working_directory);-  EXPECT_GT(size, 0);-  EXPECT_LT(size, MAX_PATH);+  EXPECT_GT(size, 0U);+  EXPECT_LT(size, (DWORD)MAX_PATH);

per google C++ style guide "Use C++-style casts like static_cast<float>(double_value), or brace initialization for conversion of arithmetic types like int64_t y = int64_t{1} << 42. Do not use cast formats like (int)x unless the cast is to void. You may use cast formats like T(x) only when T is a class type."

stephenatwork

comment created time in 5 days

more