profile
viewpoint

adonovan/gopl.io 3954

Example programs from "The Go Programming Language"

google/skylark 1145

Skylark in Go: the Skylark configuration language, implemented in Go [MOVED to go.starlark.net]

google/starlark-go 865

Starlark in Go: the Starlark configuration language, implemented in Go

google/re2j 844

linear time regular expression matching in Java

alandonovan/gopl.io 1

Example programs from "The Go Programming Language"

alandonovan/rules_haskell 0

Haskell rules for Bazel.

issue commentgoogle/starlark-go

skylark: don't access runtime.stringHash

Thanks. We could start using it in a tagged file, but we can't stop using the old hacky mechanism until two releases have passed.

alandonovan

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

+ar = [i for i in range(100000)]+target = 33333++def bench_binary_search():+    st, en = 0, len(ar) - 1

start, end

Better: lo, hi

Quarz0

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

 java_binary(     ], ) +[+    jmh_java_benchmarks(

There's no need to create one build target per file in benchmarks/*. The Java code is 100% identical for all of them. Instead just pass the list of filenames to a single Java program and do a loop in Java.

Quarz0

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

+def jmh_java_benchmarks(name, srcs, deps=[], tags=[], plugins=[], main_class="org.openjdk.jmh.Main", **kwargs):

Within Google's code base, we have a rule called java_jmh_benchmarks which supports name, srcs, deps, and tags (though apparently not plugins yet). Could you use that name, to make it easier for us to converge the two sets of rules on a single open-source one?

Quarz0

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

+ar = [i for i in range(100000)]+target = 33333++def bench_binary_search():+    st, en = 0, len(ar) - 1+    for i in range(1000000000):+        if st > en:+            break+        mid = (st + en) // 2+        if ar[mid] == target:

compute ar[mid] once

Quarz0

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

+ar = [i for i in range(100000)]+target = 33333++def bench_binary_search():+    st, en = 0, len(ar) - 1+    for i in range(1000000000):

Is this a poor man's infinite loop?

Quarz0

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

+ar = [i for i in range(100000)]

array = list(range(10000)) or just range(10000)

Quarz0

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

 public FuncallException(String msg) {    private final int numPositionalArgs; -  FuncallExpression(Expression function, ImmutableList<Argument.Passed> arguments) {+  public FuncallExpression(Expression function, ImmutableList<Argument.Passed> arguments) {

The lib.syntax package has changed a lot in the last few months, so you'll need to sync up.

Quarz0

comment created time in 9 days

Pull request review commentbazelbuild/bazel

Add JMH benchmarking tool for Starlark

 java_binary(     ], ) +[+    jmh_java_benchmarks(+        name = "Bench_" + file.replace('.', '_'),+        srcs = ["Bench.java"],+        main_class = "com.google.devtools.starlark.Bench",+        deps = [+            "//src/main/java/com/google/devtools/build/lib:events",+            "//src/main/java/com/google/devtools/build/lib:packages-internal",

Not sure why this dependency is needed. Perhaps it no longer is?

Quarz0

comment created time in 9 days

pull request commentbazelbuild/bazel

fix cpu profiler compile on ppc64le

If you can find a condition target for bare Linux (sans architecture), feel free to use it, but I didn't see one. (In any case, the jni rule logically belongs in the Bazel Java rules, not here.)

clnperez

comment created time in 12 days

issue commentbazelbuild/bazel

building bazel libcpu_profiler fails to find jni_md.h (ppc64le)

Take a look at the BUILD.oss file in the linked commit: https://github.com/bazelbuild/bazel/commit/314cd3353b018420598188cf54c3f97358baa2e3#diff-96bd71d4e041b52d389d631025d2c09fR167 It clearly doesn't have a case for linux on non-x86 platforms, so this error is expected, at least for now. You could fix it by adding a case for ppc to that select, but really there needs to be a cc_library rule for the JNI package so that Java clients don't need to think about such details.

clnperez

comment created time in 13 days

issue commentbazelbuild/bazel

onsigprof not completely async-safe

It's possible, but vanishingly unlikely for the error codes returned by write. Even strerror (no _r) should be fine in this respect, and that's what I propose to use in the fix. The very worst outcome is a rare data race leading to a finite amount of junk or the wrong error message.

benjaminp

comment created time in 16 days

issue commentgolang/go

proposal: Go 2: language: make slices of constant strings constant if the indexes are constant

The degree to which a strict reading of backward compatibility leaves little room to maneuver is constantly surprising.

ainar-g

comment created time in 17 days

issue commentbazelbuild/bazel

Crash for --compilation_mode in transition

FunctionTransitionUtil.buildSettings constructs a Dict whose values are options, which are not legal Starlark values, so it must be changed. (Also, the type of the argument passed to Starlark.call should be Map not Dict, as 'call' does not implicitly convert values as if by Starlark.fromJava. The actual value is in fact a Dict, at least for now, but we should enforce this in the Java types.)

The doc for the transition implementation function says:

"This function always has two parammeters [sic]: settings and attr. The settings param is a dictionary whose set of keys is defined by the inputs parameter. So, for each build setting --//foo=bar, if inputs contains //foo, settings will have an entry settings['//foo']='bar'."

From this description, I would expect the dict values to be strings of text as they appeared on the command line. Whether or not this was the intent, the documentation should be made more explicit. Similarly, the documentation of the return value should also be clarified with respect to the dict element types, and the meaning of the dict-of-dict case for split transitions.

moroten

comment created time in 20 days

delete branch google/starlark-go

delete branch : go-starlark-nil

delete time in 20 days

push eventgoogle/starlark-go

alandonovan

commit sha 6677ee5c7211380ec7e6a1b50dc45287e40ca9e1

clarify discussion of nil (#262) * clarify discussion of nil * fix typo

view details

push time in 20 days

push eventgoogle/starlark-go

alandonovan

commit sha a946322f2534533a170db3e5e3328a3f0faf6982

fix typo

view details

push time in 20 days

PR opened google/starlark-go

clarify discussion of nil
+4 -3

0 comment

1 changed file

pr created time in 20 days

create barnchgoogle/starlark-go

branch : go-starlark-nil

created branch time in 20 days

push eventgoogle/starlark-go

Paul Cody Johnston

commit sha ad61ea4345b9c1a718f4cb142493dc7f57176ccc

Add godoc documentation badge (#261) Reduce cognitive burden to find documentation link

view details

push time in 20 days

PR merged google/starlark-go

Add godoc documentation badge

Reduce cognitive burden to find documentation link

+1 -0

0 comment

1 changed file

pcj

pr closed time in 20 days

issue closedgoogle/starlark-go

Untyped?

starlark-go README calls Starlark an "untyped dynamic language." I'm a bit confused about why it is untyped since types are clearly there. https://github.com/bazelbuild/starlark calls it "dynamically typed language," which sounds closer to the truth.

closed time in 22 days

AlekSi

issue commentgoogle/starlark-go

Untyped?

The meaning of the term "type" has been debated for nearly a century, and there are several different viewpoints. To a logician, a type is a collection of values over which a function is defined. To a runtime implementer, a type is a value that describes the structure and operations of another value. To a compiler writer or type theorist, a type is a static classification of expressions by the values they compute, that may be used to prove the absence of certain errors. The documentation alludes to this last meaning.

I don't think "dynamically typed" is a particularly useful description as there are exceptionally few languages that are not dynamically typed. (The only examples I can think of are B and assembly languages, whose sole data type is 'machine word'.)

AlekSi

comment created time in 22 days

pull request commentgoogle/starlark-go

resolve: allow if/for/while statements at toplevel

This PR is really a request to change the spec to match reality: top-level if/for are permitted in all dialects except for Bazel, which imposes various unique restrictions on BUILD and .bzl files, this being one of them.

alandonovan

comment created time in 23 days

issue commentgoogle/starlark-go

Is there any way to stop a running script with a timeout?

Yes, I think it's essentially infeasible. An implementation in C++ could in principle impose limits on memory usage, but it would be a major usability challenge.

Like I said before, let the operating system help you. It has mechanisms to bound both CPU and memory usage of child process.

Dreamacro

comment created time in 25 days

pull request commentgoogle/starlark-go

Move Travis CI badge to the top of file [skip ci]

On Mon, 27 Jan 2020 at 10:15, Misha Brukman notifications@github.com wrote:

Yes, there are many Go Report Card findings about not using underscores, comments for exported names, and consistency in receiver names, e.g.,

Line 190: warning: receiver name x should be consistent with previous receiver name i for Int (golint)

Do you think these are worth fixing?

No. Most golint findings are a waste of time.

If you don't care about any of those, maybe it doesn't make sense to add

the Go Report Card badge. Or just add it with image-only, but without the link so it's easy to tell if it drops from A+ to A (at which point it's something to look into), but otherwise ignore? Up to you; let me know what you prefer.

The GoDoc reference badge seems useful and uncontroversial and shouldn't cause any additional effort, so that seems like a net win. Thoughts?

I have no strong feelings yet about Go Report Card, but the moment we start to game the metric by making a pointless (or harmful) change to the source just to pacify the static analysis tool is the day I delete the badge. :)

mbrukman

comment created time in a month

pull request commentgoogle/starlark-go

Move Travis CI badge to the top of file [skip ci]

Sure, sounds good to me, so long as the "go report card" widget doesn't force us to engage with all the trivial findings of golint.

mbrukman

comment created time in a month

push eventgoogle/starlark-go

Misha Brukman

commit sha 451bf671df237b29fe81813614d731a0a00d2ca7

Move Travis CI badge to the top of file [skip ci] (#257) Travis CI is more of a status badge, not an aspect of documentation, and is often kept at the top of the project's README, together with other badges, such as Go Report Card, etc., which don't belong in the "documentation" section.

view details

push time in a month

PR merged google/starlark-go

Move Travis CI badge to the top of file [skip ci]

Travis CI is more of a status badge, not an aspect of documentation, and is often kept at the top of the project's README, together with other badges, such as Go Report Card, etc., which don't belong in the "documentation" section.

+2 -2

1 comment

1 changed file

mbrukman

pr closed time in a month

pull request commentgoogle/starlark-go

Fix ineffective assignments

Thanks!

mbrukman

comment created time in a month

push eventgoogle/starlark-go

Misha Brukman

commit sha 40bc3a53d3d5f2dba8e6a22bd95a4e1a4122614b

Fix ineffective assignments (#258) * `return nil, err` after assigning to `err` (rather than ignoring it) * removed initialization of a variable just before it's re-assigned Both of these issues were found in the Go Repord Card: https://goreportcard.com/report/github.com/google/starlark-go#ineffassign

view details

push time in a month

PR merged google/starlark-go

Fix ineffective assignments
  • return nil, err after assigning to err (rather than ignoring it)
  • removed initialization of a variable just before it's re-assigned

Both of these issues were found in the Go Repord Card: https://goreportcard.com/report/github.com/google/starlark-go#ineffassign

+2 -1

0 comment

1 changed file

mbrukman

pr closed time in a month

Pull request review commentgoogle/starlark-go

Fix ineffective assignments

 func readSource(filename string, src interface{}) ([]byte, error) { 		data, err := ioutil.ReadAll(src) 		if err != nil { 			err = &os.PathError{Op: "read", Path: filename, Err: err}+			return nil, err

Good catch. I suspect no-one ever passes a Reader here which is why it went unnoticed.

mbrukman

comment created time in a month

pull request commentgoogle/starlark-go

Also run tests with Go 1.12.x and 1.13.x

Thanks!

mbrukman

comment created time in a month

push eventgoogle/starlark-go

Misha Brukman

commit sha c14ca4942ad0bd0e96d1e404ee6ac0f175fb1187

Also run tests with Go 1.12.x and 1.13.x (#259) The current setup only runs tests with Go 1.11.x and the latest source code on the `master` branch, but is not running the more recent releases 1.12.x and 1.13.x which may be widely in use. This change includes both of those versions as well.

view details

push time in a month

PR merged google/starlark-go

Also run tests with Go 1.12.x and 1.13.x

The current setup only runs tests with Go 1.11.x and the latest source code on the master branch, but is not running the more recent releases 1.12.x and 1.13.x which may be widely in use. This change includes both of those versions as well.

+2 -0

0 comment

1 changed file

mbrukman

pr closed time in a month

Pull request review commentbazelbuild/bazel

Do glob result sorting and @-escaping at the last moment.

       throw new EvalException(loc, "illegal argument in call to glob", e);     } -    return StarlarkList.copyOf(thread.mutability(), matches);+    String[] result = new String[matches.size()];+    for (int i = 0; i < result.length; i++) {+      String match = matches.get(i);+      if (match.charAt(0) == '@') {+        // Add explicit colon to disambiguate from external repository.+        match = ":" + match;+      }+      result[i] = match;+    }+    Arrays.sort(result);++    return StarlarkList.wrap(thread.mutability(), result);

Please do not make wrap public. The arrays referenced by Tuple and StarlarkList must be completely encapsulated, like ImmutableList. You must incur one copy; copyOf(Arrays.asList(x)) is optimal.

benjaminp

comment created time in a month

Pull request review commentgolang/tools

go/ssa/interp: fix goroutines and channels leaks

 func (it *stringIter) next() tuple { 	return okv } -type mapIter chan [2]value+type mapIter struct {+	iter *reflect.MapIter+	ok   bool+}++func (it *mapIter) next() tuple {+	it.ok = it.iter.Next()+	if !it.ok {+		return []value{false, nil, nil}+	}+	k, v := it.iter.Key().Interface(), it.iter.Value().Interface()+	return []value{true, k, v}+} -func (it mapIter) next() tuple {-	kv, ok := <-it-	return tuple{ok, kv[0], kv[1]}+type hashmapIter struct {+	iter *reflect.MapIter+	ok   bool+	cur  *entry+}++func (it *hashmapIter) next() tuple {+	if !it.ok {+		return []value{false, nil, nil}+	}+	if it.cur == nil {+		it.ok = it.iter.Next()+		it.cur = it.iter.Value().Interface().(*entry)

If it.iter is exhausted, then it.ok==false and it.cur==null here and the next line will fail. I think you need to check it.ok and return false before this line.

vovapi

comment created time in a month

CommitCommentEvent
CommitCommentEvent
CommitCommentEvent
CommitCommentEvent
CommitCommentEvent
CommitCommentEvent

Pull request review commentbazelbuild/bazel

GH-10176 Updating struct.to_json to escape the key string.

 private void printJson(Object value, StringBuilder sb, Location loc, String cont     }   } -  private String jsonEscapeString(String string) {-    return escapeDoubleQuotesAndBackslashesAndNewlines(string)+  private void jsonEscapeString(StringBuilder stringBuilder, String string, boolean appendColon) {+    stringBuilder.append("\"");+    stringBuilder.append(escapeDoubleQuotesAndBackslashesAndNewlines(string)         .replace("\r", "\\r")-        .replace("\t", "\\t");+        .replace("\t", "\\t"));+    if (appendColon) {

The appendColon logic has no business being in here, as it can be composed separately an no extra cost, resulting in a simpler function.

dwhu

comment created time in a month

Pull request review commentbazelbuild/bazel

GH-10176 Updating struct.to_json to escape the key string.

 private void printJson(Object value, StringBuilder sb, Location loc, String cont     }   } -  private String jsonEscapeString(String string) {-    return escapeDoubleQuotesAndBackslashesAndNewlines(string)+  private void jsonEscapeString(StringBuilder stringBuilder, String string, boolean appendColon) {

Let's call this function appendJSONStringLiteral, the StringBuilder 'out', and the String s.

dwhu

comment created time in a month

Pull request review commentbazelbuild/bazel

GH-10176 Updating struct.to_json to escape the key string.

 private void printJson(Object value, StringBuilder sb, Location loc, String cont     }   } -  private String jsonEscapeString(String string) {-    return escapeDoubleQuotesAndBackslashesAndNewlines(string)+  private void jsonEscapeString(StringBuilder stringBuilder, String string, boolean appendColon) {+    stringBuilder.append("\"");

If you like, use append('"') to avoid the internal loop.

dwhu

comment created time in a month

Pull request review commentbazelbuild/bazel

GH-10176 Updating struct.to_json to escape the key string.

 java_library(         "//third_party:guava",         "//third_party:jsr305",         "//third_party/protobuf:protobuf_java",+        "//third_party:apache_commons_lang"

This is a new dependency, and it's totally unnecessary. There is already code in StructImpl.java to escape Java strings into JSON notation (see L319); please re-use it.

dwhu

comment created time in a month

Pull request review commentbazelbuild/bazel

GH-10176 Updating struct.to_json to escape the key string.

 private void printJson(Object value, StringBuilder sb, Location loc, String cont           throw new EvalException(loc, errorMessage);         }         sb.append("\"");-        sb.append(entry.getKey());+        sb.append(StringEscapeUtils.escapeJava(String.valueOf(entry.getKey())));

Could you also apply escaping to the field names on L280? Field names should be valid identifiers, but this is not guaranteed, and we don't want to emit corrupt JSON in that case.

At that point, there are three places that emit quote escape(s) quote; could you refactor jsonEscapeString to accept a StringBuilder and emit the quotes too? Thanks.

dwhu

comment created time in a month

Pull request review commentbazelbuild/bazel

GH-10176 Updating struct.to_json to escape the key string.

 private void printJson(Object value, StringBuilder sb, Location loc, String cont           throw new EvalException(loc, errorMessage);         }         sb.append("\"");-        sb.append(entry.getKey());+        sb.append(StringEscapeUtils.escapeJava(String.valueOf(entry.getKey())));

Don't use String.valueOf(x) here; it's equivalent to x.toString() except when x is null, which can't happen because this statement is dominated by the check on L291 (and even if it could, I would prefer to see a crash in that case than to silently emit incorrect JSON).

Use: (String) entry.getKey()

dwhu

comment created time in a month

issue commentbazelbuild/starlark

spec: is dict[k] an error if k is unhashable?

More weirdness: are functions hashable or not?

% starlark
Welcome to Starlark (go.starlark.net)
>>> {len: 1}
{<built-in function len>: 1}
>>> len in {}
False
>>> {}.get(len, False)
False

% python2
Python 2.7.17rc1 (default, Oct 10 2019, 10:26:01) 
>>> {len: 1}
{<built-in function len>: 1}
>>> len in {}
False
>>> {}.get(len, False)
False


% python3.8
Python 3.8.1 (default, Dec 31 2019, 14:30:41) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> {len: 1}
{<built-in function len>: 1}
>>> len in {}
False
>>> {}.get(len, False)
False

% blaze-bin/third_party/bazel/src/main/java/com/google/devtools/starlark/Starlark
>> {len: 1}
unhashable type: 'function'
>> len in {}
unhashable type: 'function'
>> {}.get(len, False)
unhashable type: 'function'
alandonovan

comment created time in a month

issue commentbazelbuild/starlark

Formatted string literals (PEP 498)

Nice though this syntax may be, I never thought to myself while writing Python that it needs yet a third way of formatting strings. The implementation is decidedly non-trivial, affecting the scanner, parser, resolver, compiler, and runtime. Because f-strings count as variable references, the syntax tree would need to record both the literal f-string (for syntactic tools like formatters) and its parse tree (for semantic tools, like the compiler) in the syntax tree. The kludge by which x[y] appearing within a string is recursively parsed as x["y"] is quite subtle.

arturdryomov

comment created time in a month

issue openedbazelbuild/starlark

comprehensions temporarily rebind global variables

Yet another scoping bug in the Java implementation: comprehensions save and restore the existing binding but in between they clobber it, and this clobbering is visible:

$ cat a.star
x = 1
def f():
    return x
print([f() for x in [2]])

$ python2 a.star
[2]
$ python3.8 a.star
[1]
$ starlark a.star
[1]
$ blaze-bin/third_party/bazel/src/main/java/com/google/devtools/starlark/Starlark a.star
[2]

The solution is static scope resolution.

A related comprehension scoping issue: #84

created time in 2 months

pull request commentgoogle/starlark-go

Add syntax highlighting to the readme file

Thanks!

arturdryomov

comment created time in 2 months

push eventgoogle/starlark-go

Artur Dryomov

commit sha caa3e9aa50080802185728ab0d08875959c50f16

Add syntax highlighting to the readme file (#255)

view details

push time in 2 months

PR opened tweag/rules_haskell

haskell: don't call cc_toolchain.ar_executable et al

The attributes of cc_toolchain, such as ar_executable and various others, were anomalously both fields and methods, so that expressions of the form x.f and x.f() had the same value. This was a bug in Bazel that will soon be fixed, causing the x.f() form to no longer be valid.

This change replaces x.f() with x for attributes of toolchain.

+7 -7

0 comment

2 changed files

pr created time in 2 months

push eventalandonovan/rules_haskell

Alan Donovan

commit sha d119988e05d07196abf902e772d31ac6f0ccc92f

haskell: don't call cc_toolchain.ar_executable et al The attributes of cc_toolchain, such as ar_executable and various others, were anomalously both fields and methods, so that expressions of the form x.f and x.f() had the same value. This was a bug in Bazel that will soon be fixed, causing the x.f() form to no longer be valid. This change replaces x.f() with x for attributes of toolchain. Change-Id: I9eb5853c34a90d1c616d4d0c7f7f2bc5b632af79

view details

push time in 2 months

push eventgoogle/starlark-go

alandonovan

commit sha 9fcb808a6221c80a0ffa643b0be37b7d86c1d6cd

spec: document string conversion differences

view details

push time in 2 months

issue openedbazelbuild/bazel

instantiation of non-exported repository_rules is not an error

repository_rules, like package rules, should not be instantiated until they have been "exported".

"Exported" means assigned a globally unique name consisting of the pair (label, name) where label is the build label //a:b of the defining file, and name is the name of the global variable in that file that refers to the repository_rule. (The moment of export is supposed to be at the end of execution of the .bzl file, but in fact is immediately after a top-level assignment statement such as r = rule(...); but that's a story for another GitHub issue.)

Unfortunately, the check that exists for package rules was not implemented for repository_rules, so a small number of Bazel projects create a new class of repository_rule and then immediately instantiate it, without an intervening "export", as in this example:

def _foo_impl(ctx): ... 

def f():
    r = repository_rule(implementation = _foo_impl)
    r(name="myrepo", ...)

The correct Bazel behavior for this program should be an error: line 5: cannot instantiate non-exported repository_rule declared on line 4. Instead, today, it instantiates a rule, and gives it the name "myrepo" and the rule class name "r". Bazel infers the rule class name using a hack: it looks at the syntax tree of the r(...) call expression and extracts the name "r". (This heuristic would fail on a more complex call expression such as [r][0](...).)

The resulting name is observable both in the user interface, and to Starlark programs, for example through the result of native.existing_rules() when called in a WORKSPACE thread.

This hack is an obstacle to a Starlark compiler, as we cannot have built-in functions looking through the syntax tree during execution. We plan need to change the heuristic to something else (e.g. to use a name such as "unexported-_foo_impl" instead of r.

created time in 2 months

issue commentbazelbuild/bazel

crash passing a function as attr.int's default

Fixed by internal CL 286037108.

benjaminp

comment created time in 2 months

issue commentbazelbuild/bazel

crash passing a function as attr.int's default

You're absolutely right; I've been making many changes to lib/syntax recently and this looked like plausible fallout from one of them. I have a fix pending.

benjaminp

comment created time in 2 months

issue commentbazelbuild/bazel

crash passing a function as attr.int's default

I can reproduce it; looking into a fix now. Sorry for the breakage.

benjaminp

comment created time in 2 months

PR closed adonovan/gopl.io

Duplicate text and count does not print

This example does not appear to print the text and count to the console. For reference, I am using version 1.12.7 of the go language on osx. If the logic that iterates over the map of counts is moved inside the standard input scanner for loop then I receive the appropriate log statements.

+6 -5

1 comment

1 changed file

rjwehrle

pr closed time in 2 months

pull request commentadonovan/gopl.io

Duplicate text and count does not print

The print statement won't be reached until the scanner loop has finished, which means the end-of-file has been reached when reading from os.Stdin. Are you running the program interactive so that it reads from the terminal? If so, then you'll need to close the program's stdin by typing Ctrl-D when you've finished entering text.

rjwehrle

comment created time in 2 months

issue openedbazelbuild/bazel

starlark: parser requires newline (not semicolon) after loads statement

The parser in the Java implementation of Starlark appears to require a newline, and does not accept a semicolon, after a load statement.

$ blaze run third_party/bazel/src/main/java/com/google/devtools/starlark/Starlark
>> load("foo", "f"); f()
.. 
ERROR :1:17: syntax error at ';': expected newline

created time in 3 months

issue commentbazelbuild/starlark

default argument of getattr

Done. https://github.com/google/starlark-go/commit/52794b6d230111cdb5f3f45816f441da67c09926

laurentlb

comment created time in 3 months

pull request commentgoogle/starlark-go

.travis.yml: test that module is tidy

LGTM I guess?

jayconrod

comment created time in 3 months

push eventgoogle/starlark-go

Koichi Shiraishi

commit sha a9ae2c9c54732bd8c4ed07395bd891373a571e83

go.mod: init Go module (#243)

view details

push time in 3 months

PR merged google/starlark-go

go.mod: init Go module

Initialize Go module support.

Close: #232

+18 -0

4 comments

2 changed files

zchee

pr closed time in 3 months

issue closedgoogle/starlark-go

go.mod

This would be quite useful to add.

closed time in 3 months

andrewchambers

pull request commentgoogle/starlark-go

go.mod: init Go module

Got it, thanks Jay.

zchee

comment created time in 3 months

push eventgoogle/starlark-go

alandonovan

commit sha 52794b6d230111cdb5f3f45816f441da67c09926

spec: document 3-arg getattr Change-Id: I0d547e0cdc908b89250115805925f83fa2d87714

view details

push time in 3 months

issue commentbazelbuild/bazel

x.f(g()) evaluates g() before x.f

This optimization also hides other cases in which x.f() cannot be decomposed into y= x.f; y() without changing its behavior, as in this example which accesses struct.to_json:

load("inc.bzl", "f")
f()

$ cat test/inc.bzl 
def f():
    print(struct(x = 1).to_json())

    y = struct(x = 1).to_json; print(y)

$ blaze query test/BUILD
DEBUG: test/inc.bzl:2:5: {"x":1}
ERROR: test/inc.bzl:4:9: Traceback (most recent call last):
	File "test/BUILD", line 3
		f()
	File "test/inc.bzl", line 4, in f
		struct(x = 1).to_json
'struct' object has no attribute 'to_json'
Available attributes: x
alandonovan

comment created time in 3 months

issue openedbazelbuild/bazel

x.f(g()) evaluates g() before x.f

Bazel's implementation of Starlark evaluates x.f(g()) in the wrong order: it evaluates the argument g() before the field selection operation x.f. This causes the side effects of g to be observed even when x.f fails, which is confusing. In principle, the effects of g could even influence the behavior of the x.f operation.

$ cat a.sky
def f(): print("f")
"".nonesuch(f())

$ blaze-bin/third_party/bazel/src/main/java/com/google/devtools/starlark/Starlark  a.sky
f
type 'string' has no method nonesuch()

Contrast:
$ python2 a.sky
Traceback (most recent call last):
  File "a.sky", line 4, in <module>
    "".nonesuch(f())
AttributeError: 'str' object has no attribute 'nonesuch'

$ python3 a.sky
Traceback (most recent call last):
  File "a.sky", line 4, in <module>
    "".nonesuch(f())
AttributeError: 'str' object has no attribute 'nonesuch'

$ starlark a.sky # in Go
Traceback (most recent call last):
  a.sky:4:3: in <toplevel>
Error: string has no .nonesuch field or method

(The cause of this behavior is an unsound optimization to avoid allocating a closure for x.f.)

created time in 3 months

issue openedbazelbuild/starlark

x.f(g()) evaluates g() before x.f

Bazel's implementation of Starlark evaluates x.f(g()) in the wrong order: it evaluates the argument g() before the field selection operation x.f. This causes the side effects of g to be observed even when x.f fails, which is confusing. In principle, the effects of g could even influence the behavior of the x.f operation.

$ cat a.sky
def f(): print("f")
"".nonesuch(f())

$ blaze-bin/third_party/bazel/src/main/java/com/google/devtools/starlark/Starlark  a.sky
f
type 'string' has no method nonesuch()

Contrast:
$ python2 a.sky
Traceback (most recent call last):
  File "a.sky", line 4, in <module>
    "".nonesuch(f())
AttributeError: 'str' object has no attribute 'nonesuch'

$ python3 a.sky
Traceback (most recent call last):
  File "a.sky", line 4, in <module>
    "".nonesuch(f())
AttributeError: 'str' object has no attribute 'nonesuch'

$ starlark a.sky # in Go
Traceback (most recent call last):
  a.sky:4:3: in <toplevel>
Error: string has no .nonesuch field or method

(The cause of this behavior is an unsound optimization to avoid allocating a closure for x.f.)

created time in 3 months

pull request commentgoogle/starlark-go

go.mod: init Go module

@jayconrod Jay, can you review this, as you are likely the world expert on Go modules?

Thanks @zchee and for the immense delay.

zchee

comment created time in 3 months

push eventgoogle/starlark-go

alandonovan

commit sha 698e61cae82e899f5597668f27cb312153f2da85

doc: three minor tweaks to spec Change-Id: If04ab510bf0746191e05a55f4fa2911b36e845c9

view details

push time in 3 months

issue commentbazelbuild/bazel

depset: depset(direct=...) constructor does not check element validity

Without the flag, depset([(x,)]) reports an error if and only if x is (or contains) a mutable value. With the flag, the expression succeeds even if x is (or contains) a mutable value. I could not see any difference between Blaze at tip and the previous release of Blaze, which is encouraging.

alandonovan

comment created time in 3 months

issue commentgolang/go

cmd/vet: vet flags for specifying 'analyzer' doesn't work

Thanks, this does indeed look like a bug. Please do share your fix.

$ go version
go version devel +4a378d712d Tue Nov 26 20:21:04 2019 +0000 linux/amd64
$ cd $HOME
$ go vet -unreachable=true runtime/testdata/testprog
# runtime/testdata/testprog
go-again/src/runtime/testdata/testprog/deadlock.go:45:2: unreachable code
go-again/src/runtime/testdata/testprog/deadlock.go:50:2: unreachable code
$ go vet -unreachable=false runtime/testdata/testprog
# runtime/testdata/testprog
go-again/src/runtime/testdata/testprog/deadlock.go:45:2: unreachable code
go-again/src/runtime/testdata/testprog/deadlock.go:50:2: unreachable code
shawn-xdji

comment created time in 3 months

issue openedbazelbuild/bazel

depset: depset(direct=...) constructor does not check element validity

The depset(direct=...) constructor that we have been recently migrating users towards does not apply the same element validity checks as the legacy depset([...]) constructor. Consequently one can insert mutable items such as structs and tuples containing lists:

x = [1, 2, 3]
s = depset(direct = [(x,)])
print(s)  # 'depset([([1, 2, 3],)])'

x.append(4)  # 'depset([([1, 2, 3, 4],)])'
print(s)

The fix is to apply the same checks as done by the legacy constructor, but this is a breaking change. The new check will this be behind a flag called --incompatible_always_check_depset_elements.

created time in 3 months

issue commentbazelbuild/bazel

depset hashability requirement is a breaking change

Google issue number: b/144992997

benjaminp

comment created time in 3 months

issue commentbazelbuild/bazel

depset hashability requirement is a breaking change

To clarity, my original change had two effects, one significant, one not: First, it changed the element validity check from "is immutable" to "is hashable". This change does affect change the error message, but is otherwise relatively minor. Second, it copied the element validity check current applied to the depset([x]) constructor to the depset(direct=[x]) constructor as well. This fixed a bug that existing programs currently exploit, and is thus a breaking change.

benjaminp

comment created time in 3 months

issue commentgoogle/starlark-go

Is there any way to stop a running script with a timeout?

Oh, if you can't trust the user's function then you definitely should not run it in the same address space as your code, as it can trivial consume all available memory and there's nothing we can hope to do about that.

Let the operating system help you. Fork and exec a subprocess in a cgroup or setrlimit container that bounds the memory, CPU, and running time.

Dreamacro

comment created time in 3 months

issue commentgoogle/starlark-go

Is there any way to stop a running script with a timeout?

Currently there is no way to stop a function in the middle of Starlark execution, though it is a feature that has been requested before and may be feasible to implement without a major performance penalty. Is the function you wish to interrupt executing a Starlark loop, or is it calling into a built-in function that takes a long time? If the latter, then you can easily make that built-in function honor the cancellation of a context carried within the thread. For example:

thread := &starlark.Thread{Name:  "example"}
ctx, cancel := context.WithTimeout(context.background(), time.Second)
defer cancel()
thread.SetLocal("context", ctx)
starlark.ExecFile(...)

...

func slowBuiltin(thread *Thread, fn *Builtin, args Tuple, kwargs []Tuple) (Value, error) {
    if (thread.Local("context").(context.Context).Err() != nil) { ... give up ... }
    ...
}
Dreamacro

comment created time in 3 months

push eventgoogle/starlark-go

Brandon Bercovich

commit sha aaf7be0038921cd52ad0b7430265094afe746042

Adding support for Bazel test to use DataFile (#251)

view details

push time in 3 months

PR merged google/starlark-go

Adding support for Bazel test to use DataFile

https://github.com/google/starlark-go/issues/250

+9 -0

0 comment

1 changed file

bbercovich

pr closed time in 3 months

Pull request review commentgoogle/starlark-go

Adding support for Bazel test to use DataFile

 func freeze(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, k // 'go build', under which a test runs in its package directory, // and Blaze, under which a test runs in the root of the tree. var DataFile = func(pkgdir, filename string) string {+	// Check if we're being run by Bazel and change directories if so.+	// TEST_SRCDIR and TEST_WORKSPACE are set by the Bazel test runner, so that makes a decent check+	testSrcdir := os.Getenv("TEST_SRCDIR")+	testWorkspace := os.Getenv("TEST_WORKSPACE")+	if testSrcdir != "" && testWorkspace != "" {+		return filepath.Join(testSrcdir, "net_starlark_go", pkgdir, filename)

OK, thanks.

bbercovich

comment created time in 3 months

Pull request review commentgoogle/starlark-go

Adding support for Bazel test to use DataFile

 func freeze(thread *starlark.Thread, _ *starlark.Builtin, args starlark.Tuple, k // 'go build', under which a test runs in its package directory, // and Blaze, under which a test runs in the root of the tree. var DataFile = func(pkgdir, filename string) string {+	// Check if we're being run by Bazel and change directories if so.+	// TEST_SRCDIR and TEST_WORKSPACE are set by the Bazel test runner, so that makes a decent check+	testSrcdir := os.Getenv("TEST_SRCDIR")+	testWorkspace := os.Getenv("TEST_WORKSPACE")+	if testSrcdir != "" && testWorkspace != "" {+		return filepath.Join(testSrcdir, "net_starlark_go", pkgdir, filename)

I'm curious: is this net_starlark_go name produced by Bazel when it sets up the repository for a build, or is it a name you chose?

bbercovich

comment created time in 3 months

issue commentgolang/go

x/tools/cmd/stringer: stringer should not use go/types

Stringer continues to use go/types, and uses go/packages to obtain package information and export data for dependencies. Just now it took about 700ms to execute the Pill example in its documentation (a single source file with no imports), compared to about 60ms to compile that package, so stringer is indeed much slower than compilation. Most of that time appears to be spent in go/packages, since the golang.org/x/tools/go/packages/gopackages -mode=types command takes about the same amount of time. I'm not sure why go/packages takes so long to do ~nothing. @matloob @ianthehat ?

robpike

comment created time in 3 months

issue commentgolang/go

cmd/vet: warn about string(int) type conversions

That could be invalid UTF-8. So string(buf[x:y]) may be a bug.

It could be, but it very much depends on the situation.

To extract strings from buffers, should we generally use string(bytes.ToValidUTF8(buf, nil))?

I had never even heard of that function until now, and I'm more deliberate than most about string encodings. Slicing a string or byte slice may be perfectly fine, again depending on the situation.

ianlancetaylor

comment created time in 4 months

issue commentgolang/go

cmd/vet: warn about string(int) type conversions

string([]byte{b}) is not equivalent to string(b): the former yields the string whose bytes are just b, the latter returns the string that UTF-8-encodes the single rune b. If b is greater than 127, then the former will be a single-byte string byte the latter will be a two-byte encoding.

it would still be a nuisance to have to update all my code to use this workaround :(

Michael Matloob recently presented at a conference the "suggested fixes" feature of the analysis Diagnostics API. The diagnostic for the new analyzer could replace (or offer to replace) the deprecated expressions by the preferred version.

ianlancetaylor

comment created time in 4 months

issue commentgoogle/starlark-go

Support keyword-only args in UnpackArgs (i.e. for Built-Ins)

Hi Dan, sorry for the delay. Yes, this would be a nice feature, as most application built-ins seem to want most of their parameters to be keyword-only.

Currently we encode one extra bit into the name string: "x" means a mandatory parameter and "x?" means an optional one. Perhaps we could use "x=" to mean a mandatory keyword-only parameter and "x=?" to mean optional keyword only. This is only the start of a design though; we need to think through the interactions carefully. Ignoring the *args and **kwargs residuals for now, the general schema for a Python3 function consists of three tuples divided by / and *:

positional-only / positional-or-named * keyword-only

with each tuple having zero or more ident (mandatory) followed by zero or more ident=expr (optional) parameters. I wonder how close we can get to expressing the full schema by encoding extra bits in each parameter name, in the style of "x?", without excessive complexity, and ideally while remaining compatible with the existing UnpackArgs function.

dbentley

comment created time in 4 months

issue openedgoogle/starlark-go

disallow reversed(dict), and clarify spec.

Python allows any iterable sequence, but disallows dict; it returns a reverse iterator. Starlark-in-Java also disallows dict, and in addition disallows string, which is not iterable in this dialect. Starlark-in-Go also disallows string, but allows dict. This appears to be a bug.

@laurentlb

created time in 4 months

push eventadonovan/gopl.io

alandonovan

commit sha 3ccaffe2787408a91e6663d16d86fce3aada8422

Create CNAME

view details

alandonovan

commit sha 1e9cbf3a0aca1d05c33d3ab9332c2346dabf3d27

Merge pull request #36 from alandonovan/patch-1 Create CNAME

view details

push time in 4 months

PR merged adonovan/gopl.io

Create CNAME
+1 -0

0 comment

1 changed file

alandonovan

pr closed time in 4 months

PR opened adonovan/gopl.io

Create CNAME
+1 -0

0 comment

1 changed file

pr created time in 4 months

push eventalandonovan/gopl.io

alandonovan

commit sha 3ccaffe2787408a91e6663d16d86fce3aada8422

Create CNAME

view details

push time in 4 months

more