Daniel Martí mvdan Sheffield, United Kingdom I work on stuff in Go.

govim/govim 518

govim is a Go development plugin for Vim8, written in Go

GitHub Actions as CI for Go

gunk/gunk 289

Modern frontend and syntax for Protocol Buffers

mvdan/garble 188

Obfuscate Go builds

F-Droid desktop client

Manage wireless access points in Android (abandoned)

Run Go tests inside a Docker image

Minimalist BitWarden client

Benchmark the init cost of Go packages

List merged and cherry-picked branches

issue commentgolang/go

It's unclear to me if the current proposal is to memoize all strings, or just keys when decoding into map[string]T. I imagine most cases of unique/changing strings don't happen in map keys, but I still find that a possible scenario. For example, a map where the keys are unique IDs or hashes seems pretty reasonable, if one wants to then do quick lookups after unmarshaling.

So, my concern is probably smaller if we only memoize map keys, but even then, I'm still a bit concerned.

rsc

comment created time in 18 hours

issue commentgolang/go

@mvdan, are you concerned about the overhead of the map lookup in your Jun 26 comment? From the outside there can be no possible downside to reusing all strings that are constructed.

@rsc sorry that I missed your comment for this long. I could have sworn I had replied.

My concern is about keeping strings alive for longer than needed. That's very much related to one of your original points:

It's unclear to me whether the map in a Decoder should persist between Decode operations. Probably?

For example, imagine if I have a server that receives JSON objects via a network stream, unmarshals them, writes them to a database, and discards them. Right now, I would write this program in a way that the "incoming objects" goroutine keeps a single decoder alive forever.

If the decoder starts memoizing strings forever (since the decoder persists forever), my memory usage could increase over time if I keep seeing new/unique strings. For example, it seems pretty common to represent UUIDs and hashes as strings in JSON, so I'm a bit worried that users could easily run into this kind of "memory leak".

rsc

comment created time in a day

issue commentmvdan/garble

Unfortunately I don't think there is a way to do this with our design. We obfuscate one package at a time, while other tools like gobfuscate work on an entire GOPATH at once. When we're obfuscating a package foo/bar, it's already being built under that import path, because we are not in control of resolving dependencies and locating packages. We only control the compiler and linker's inputs.

I really don't want to take the "obfuscate the entire world" approach that gobfuscate takes, too. That requires a ton of upfront work and it's slow, so it doesn't scale to large builds at all. It's also much trickier to implement properly if one wants to support both GOPATH and modules.

lu4p

comment created time in a day

issue commentlibp2p/go-libp2p-swarm

Thanks for the quick fix! You might want to check the other libp2p repositories for similar lines via grep, because other similar data races might be lurking.

mvdan

comment created time in 2 days

issue commentlibp2p/go-libp2p-swarm

After using context.Background() in the constructor, in favor of the Close method alone for cancellation, the race seems to have gone away. Perhaps this falls under "working as intended", but it still got me stuck for a while.

mvdan

comment created time in 2 days

issue openedlibp2p/go-libp2p-swarm

WARNING: DATA RACE
Read at 0x00c000a32748 by goroutine 137:
github.com/libp2p/go-libp2p-swarm.(*Swarm).teardown()
/home/mvdan/go/pkg/mod/github.com/libp2p/go-libp2p-swarm@v0.2.2/swarm.go:122 +0x5c
github.com/libp2p/go-libp2p-swarm.(*Swarm).teardown-fm()
/home/mvdan/go/pkg/mod/github.com/libp2p/go-libp2p-swarm@v0.2.2/swarm.go:118 +0x41
github.com/jbenet/goprocess.(*process).doClose()
/home/mvdan/go/pkg/mod/github.com/jbenet/goprocess@v0.1.3/impl-mutex.go:233 +0x40e
github.com/jbenet/goprocess.(*process).Close()
/home/mvdan/go/pkg/mod/github.com/jbenet/goprocess@v0.1.3/impl-mutex.go:175 +0x114
github.com/jbenet/goprocess/context.CloseAfterContext.func1()
/home/mvdan/go/pkg/mod/github.com/jbenet/goprocess@v0.1.3/context/context.go:67 +0x14d

Previous write at 0x00c000a32748 by goroutine 17:
github.com/libp2p/go-libp2p-swarm.NewSwarm()
/home/mvdan/go/pkg/mod/github.com/libp2p/go-libp2p-swarm@v0.2.2/swarm.go:113 +0x734
github.com/libp2p/go-libp2p/config.(*Config).NewNode()
/home/mvdan/go/pkg/mod/github.com/libp2p/go-libp2p@v0.6.0/config/config.go:118 +0x360
[... our test and code that calls NewNode synchronously]
testing.tRunner()
/home/mvdan/tip/src/testing/testing.go:992 +0x1eb

Goroutine 137 (running) created at:
github.com/jbenet/goprocess/context.CloseAfterContext()
/home/mvdan/go/pkg/mod/github.com/jbenet/goprocess@v0.1.3/context/context.go:64 +0xb9
github.com/jbenet/goprocess/context.WithContextAndTeardown()
/home/mvdan/go/pkg/mod/github.com/jbenet/goprocess@v0.1.3/context/context.go:28 +0x208
github.com/libp2p/go-libp2p-swarm.NewSwarm()
/home/mvdan/go/pkg/mod/github.com/libp2p/go-libp2p-swarm@v0.2.2/swarm.go:112 +0x648
github.com/libp2p/go-libp2p/config.(*Config).NewNode()
/home/mvdan/go/pkg/mod/github.com/libp2p/go-libp2p@v0.6.0/config/config.go:118 +0x360
[... our test and code that calls NewNode synchronously; same line as before]
testing.tRunner()
/home/mvdan/tip/src/testing/testing.go:992 +0x1eb

Goroutine 17 (running) created at:
testing.(*T).Run()
/home/mvdan/tip/src/testing/testing.go:1043 +0x660
testing.runTests.func1()
/home/mvdan/tip/src/testing/testing.go:1312 +0xa6
testing.tRunner()
/home/mvdan/tip/src/testing/testing.go:992 +0x1eb
testing.runTests()
/home/mvdan/tip/src/testing/testing.go:1310 +0x57f
testing.(*M).Run()
/home/mvdan/tip/src/testing/testing.go:1220 +0x3a6
main.main()
_testmain.go:43 +0x219


This is a partial trace, because I assume it's enough for you to realise what's going on. My assumption is that, in the lines below in swarm.go, it's possible to read s.ctx at the same time as it's being set:

	s.proc = goprocessctx.WithContextAndTeardown(ctx, s.teardown)
s.ctx = goprocessctx.OnClosingContext(s.proc) // NOTE: racy write

return s
}

func (s *Swarm) teardown() error {
// Wait for the context to be canceled.
// This allows other parts of the swarm to detect that we're shutting
// down.


I haven't been able to craft a main.go that reproduces the race reliably, unfortunately. This only happens as part of a non-trivial test. I can provide more details privately, if you prefer.

Continuing with my assumptions - I assume we should be setting s.ctx before it's ever used in any way. I don't know the goprocessctx package well, though. At the end of the test, we call host.Close and cancel on the context used in libp2p.Config.NewNode, in that order. Perhaps that's wrong, given #59.

created time in 2 days

issue commentgolang/go

@gopherbot, please open a backport issue for 1.14.

I think this is a regression - https://go-review.googlesource.com/c/go/+/193604 was a good change, but we didn't properly think of all the edge cases. I have a fix nearly ready. CC @breml @dsnet

AllenX2018

comment created time in 2 days

push eventmvdan/sh

commit sha ba810aae1c700e264e2a1a3b9c457c8a071143a1

push time in 2 days

PR merged mvdan/sh

Hi there

I've been used the shfmt for a longe time, and since the beginning of this year, I started to validate my projects using github actions.

This PR adds a link to the project in README.md. I checked and there is no other github action do to that. Btw, I reorder the links by name.

+11 -8

0 comment

1 changed file

luizm

pr closed time in 2 days

issue commentgolang/go

You're right, @AllenX2018, something is up with this change in behavior. Can you please file a separate issue? I'm already investigating a fix, but it's good to have an issue to track the bug.

AllenX2018

comment created time in 3 days

issue commentgolang/go

I don't know, the only few times I've really needed performant string handling, I've found that writing code tailored to my needs is the easiest way. Also see https://golang.org/doc/faq#x_in_std.

codekidX

comment created time in 3 days

issue closedmvdan/sh

Example:

--- ./t.orig
+++ ./t
@@ -1,7 +1,7 @@
#!/bin/bash

if [[ -v foo ]] \
-&& [[ -v bar ]]; then
+       && [[ -v bar ]]; then
echo "$foo-$bar"
fi


In here, shfmt was run with the -bn and -i 0 args. Now, I guess that the break at the very top tells shfmt to indent the line that follows \, but those are part of the same single logic expression and shfmt was explicitly told that logic operators can start a line. Additionally, as a result, the line below gets aligned with the part of the if expression. It compares in my mind to having something like this basically:

if :; then
foo
fi


I don't say this is a bug or anything, more as a conscious formatting choice, however, I would appreciate some feedback on if it's possible for you to easily make shfmt handle scenarios like that without actually applying the indentation. :)

closed time in 3 days

mikeBashStuff

issue commentmvdan/sh

Thanks for your feedback! I definitely agree it takes some using to, but I can't think of a clearly better approach without ugly side effects. The only other possible tradeoff would be to never indent backslash continuations, but then we'd make some common use cases pretty ugly:

if foo; then
for bar in $(...); do long-command \$foo bar done fi  I'm closing this for now, but we can resurrect this thread in the future if anyone has any suggestion. mikeBashStuff comment created time in 3 days issue commentmvdan/sh Makes sense. Yeah, master and the latest releases generally require a somewhat recent version of Go, released in the past 6-12 months. mikeBashStuff comment created time in 3 days issue commentmvdan/garble I didn't think of CGO at all, nor are there any tests for it, so I'm not surprised it's broken. Patches or ideas welcome. I might take a look soon. GeeMing comment created time in 3 days issue commentmvdan/sh That was a bit long, but it was necessary to give some context :) I'm open to ideas on how we could do this better in the future. But we can't change this rule in v3 right now, because that would be a breaking change. mikeBashStuff comment created time in 3 days issue commentmvdan/sh Hm - I think this is working as intended. "can start a line" doesn't imply excluding indentation. Backslashes introduce a level of indentation, just like opening a block: foo \ bar  Additionally, as a result, the line below gets aligned with the part of the if expression. Yeah, that's the tradeoff we make. The alternative would be to support "stacked indentation", like: if [[ -v foo ]] \ && [[ -v bar ]]; then echo "foo-$bar" fi  However, this can get crazy pretty fast. The indentation level could increase by three or four levels in a single line. Instead, what we do is follow what I think is an OK tradeoff made by gofmt - each line can only increase the indentation level by one. It does lead to some perhaps confusing edge cases, though. Like you say above, some lines appear to belong to the same level/block, when they don't. For example, in Go: func foo(first int, second int) { println(first, second) }  Note how the second argument is at the same level as the body. I personally avoid this ugly edge case by avoiding newlines in function declarations or if statement conditions, and if I really must use many lines, I leave an empty line later to clarify the separation: func foo(first int, second int) { println(first, second) }  Following the same principle, this isn't super pretty, but I think it's okay for very long conditions: if [[ -v foo ]] \ && [[ -v bar ]]; then echo "$foo-$bar" fi  mikeBashStuff comment created time in 3 days issue commentmvdan/sh Yeah, it would be more helpful if this issue included a minimal reproducer with a tiny Go program, e.g. on the playground. andreynering comment created time in 3 days issue closedmvdan/sh Problematic scenario looks like the following: # shellcheck-static <(shfmt-3.0.2 "${shfmt_cmdline[@]}" ./t)

In /dev/fd/63 line 3:
for foo; do "$foo" &;done ^-- SC1045: It's not 'foo &; bar', just 'foo & bar'. # shfmt_cmdline+=(-d) # shfmt-3.0.2 "${shfmt_cmdline[@]}" ./t
--- ./t.orig
+++ ./t
@@ -1,4 +1,5 @@
#!/bin/bash

-for foo; do "$foo" & done; wait +for foo; do "$foo" &;done
+wait

# echo shfmt-3.0.2 "${shfmt_cmdline[@]}" shfmt-3.0.2 -i 0 -bn -ci -ln bash -d #  Where, unfortunately, this generates a syntax error, leaving the potential script completely unusable. Moreover, the shfmt fails in the same manner even when call to wait is moved onto its own line. The only way to avoid this is by not keeping the for loop as a one-liner, i.e., to move its done keyword to a new line. Any hints, comments would be appreciated. :) closed time in 3 days mikeBashStuff issue commentgolang/go This seems like quite a lot more API surface for little benefit. If you really care about performance, why not write this yourself by combining lower-level APIs like strings.Builder and strings.IndexByte? codekidX comment created time in 3 days issue commentgolang/go Overall, I like how refreshingly simple this proposal is, compared to most other error handling proposals. It deals with the verbosity without adding too much magic. But if you have any non-return statement in the if body, then it formats as multiple lines. I think this rule could be simplified to: this kind of if statement is allowed to fit in a single line if its body is exactly one statement, and its length is short enough. This rule is already in place in gofmt, for single-line functions. The heuristic is just not explicitly defined, because it might change in the future. See it in practice - click "format" on https://play.golang.org/p/rEyBUh5rj7V this makes error handling much lighter looking (the common motivation for new error handling proposals) I think another point in favor of the handle proposal was to consistently wrap errors in a function: // go2 draft func process(...) error { handle err { return fmt.Errorf("process: %w", err) } check doFoo() check doBar() } // this proposal func process(...) error { if err := doFoo(); err { return fmt.Errorf("process: %w", err) } if err := doBar(); err { return fmt.Errorf("process: %w", err) } return nil }  Note how we repeat the error wrapping/decoration. This is good if we want the call site to decorate an error with the function that was called, but I thought the point of the go2 draft was also to encourage functions to decorate errors with their own information when returning. bradfitz comment created time in 3 days push eventmvdan/bitw commit sha 056480d6f3ec7bb51126d4f788f70b31a3c4eb63 reject unknown config keys Otherwise, typos like "idURL" instead of "identityURL" might go unnoticed, and cause confusing errors. Fixes #11. push time in 3 days issue closedmvdan/bitw Config: email = 132@ikl.sh apiURL = https://bw.ikl.sh/api/ idURL = https://bw.ikl.sh/identity/  Output: $ bitw login


Data file:

{
"DeviceID": "[redacted]",
"AccessToken": "",
"RefreshToken": "",
"TokenExpiry": "0001-01-01T00:00:00Z",
"KDF": 0,
"KDFIterations": 100000,
"LastSync": "0001-01-01T00:00:00Z",
"Sync": {
"Profile": {
"ID": "",
"Name": "",
"Email": "",
"EmailVerified": false,
"Culture": "",
"TwoFactorEnabled": false,
"Key": "",
"PrivateKey": "",
"SecurityStamp": "",
"Organizations": null
},
"Folders": null,
"Ciphers": null,
"Domains": {
"EquivalentDomains": null,
"GlobalEquivalentDomains": null
}
}
}


closed time in 3 days

132ikl

issue commenttendermint/tm-db

Nice, thanks!

dependabot-preview[bot]

comment created time in 3 days

issue commentmvdan/bitw

You're very correct, @mk-f - thanks for noticing that :) I'll teach the tool to complain about unknown options, to prevent other typos and confusion in the future.

132ikl

comment created time in 3 days

issue commenttendermint/tm-db

@marbar3778 any chance you could fix this? It's making using newer versions of this module (such as v0.5.0) impossible, since the build system errors. The boltdb modulde has always been called this way (see https://github.com/etcd-io/bbolt/commit/e65d4d5d27e82efe8e16175a11ec2f6e8067c990#diff-37aff102a57d3d7b797f152915a6dc16).

dependabot-preview[bot]

comment created time in 3 days

issue commentgolang/go

The rules for anonymous structs (i.e. following the Go visibility rules) are the same when unmarshaling - that's why I linked them earlier. If you want to make that clearer, we can, but we should point to the existing rules (already documented) instead of explaining the same rules in a different way.

icco

comment created time in 4 days

issue commentgolang/go

whether making an unauthenticated gRPC call is considered "convenient" enough

I want to make use of this in scripts, so it should be plan HTTP+plaintext, just like the existing endpoint with mode=text. So a gRPC client won't do, I'm afraid.

whether the expectation of stability of the API based on the domain name is good enough

I'd prefer this to live under golang.org, simply because that's also where one downloads the releases from. It's also a bit more convenient (i.e. shorter) to use. Not a big problem, but if what we want is to consolidate all of this into the maintainer API, we should just deprecate the existing /dl REST endpoint.

Note that, as said before, the replacement needs to support REST and plaintext endpoints too, otherwise we'd be losing valuable features.

whether including a new minor release as soon as it is tagged is what you're looking for

I imagine a large amount of the use cases (including mine) involve using the release, i.e. downloading it. So yeah, the feature should show the latest fully released version.

mvdan

comment created time in 4 days

issue commentgolang/go

This seems like an almost exact duplicate of https://github.com/golang/go/issues/36898 :)

changkun

comment created time in 4 days

issue commentgolang/go

That makes sense, is there a common way to change this code to deal with this case?

The general answer here is don't use anonymous struct fields that introduce ambiguity. There is also json:"-" if you want to skip an entire anonymous field, to let another one be used completely.

Otherwise, it does feel like a bug because it silently fails.

There is no failure. It's just not doing what you think it should.

I wonder if it would be worth adding to go vet or something to prevent people from doing this.

The program above is valid, so vet shouldn't complain about it.

Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph.

If you think the docs could be made clearer, suggestions or patches are welcome. But there isn't a bug here, as far as I can tell.

icco

comment created time in 4 days

issue closedgolang/go

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions --> (sorry because I kown it is a stupid question)

### What did you do?

go get -u golang.org/x/tools/cmd/goyacc
go tool yacc


<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

### What did you expect to see?

something useful like usage

### What did you see instead?

go tool: no such tool "yacc"


p.s. I try to search how to run yacc in Google, but the message to too useless. I hope you can help me... Thanks

closed time in 5 days

PeterlitsZo

issue commentgolang/go

What @seankhliao said. Also see the go get docs at https://golang.org/pkg/cmd/go/internal/get/.

PeterlitsZo

comment created time in 5 days

issue commentgolang/go

I think @daenney is right, and I'm saddened to see that @ALTree's PR hasn't gotten attention. I'm sure the maintainers are busy people, and I realise there are lots of open issues and PRs, but this isn't just another PR.

flibustenet

comment created time in 5 days

issue closedgolang/go

### What did you do?

My struct has a string field with a json:"string" field tag. When this field contains escaped character, I get unexcepted result when doing roundtrip test. <!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

type T struct {
F1 string json:"F1,string"
}

t := T {
"aaa\tbbb",
}

b, _ := json.Marshal(t)
if err := json.Unmarshal(b, &t); err == nil {
fmt.Printf("%+v\n", t)
}else {
fmt.Println(err)
}


### What did you expect to see?

{F1:aaa	bbb}


### What did you see instead?

json: invalid use of ,string struct tag, trying to unmarshal "\"aaa\tbbb\"" into string


closed time in 5 days

AllenX2018

issue commentgolang/go

I've confirmed this is a duplicate of #38105. I'll close this issue to consolidate into the other one, but I'll add this as a test case.

AllenX2018

comment created time in 5 days

issue commentactions/runner

Could be worth testing with the preemptive scheduler disabled in CI?

Proving a failure positive is trivial, but proving a negative is near impossible. I guess I could have changes like that in master for a few weeks, but it seems like shots in the dark unless we actually have a working theory.

Also, note that all three failures linked above are for Go 1.13.x, so I don't think this is a recent issue with Go 1.14. I think I've had failures with 1.14, but I don't have a link for proof right now. I've disabled that test for now to avoid more headaches, so I won't be getting more failure logs for the time being.

Are you running the exact same Linux distro? Kernel etc?

Probably not. I haven't investigated further in this direction. I don't think Linux or the distro are to blame, given that Mac shows the exact same bug.

mvdan

comment created time in 5 days

issue commentgolang/go

Thanks @dsnet - I'll try to have a look at it this weekend.

AllenX2018

comment created time in 6 days

issue closedgolang/go

### What did you do?

I tried to implement encoding.TextUnmarshaler for unmarshaling JSON. However when escaped characters are added to the end of a JSON value the unmarshaled value becomes mangled and includes JSON formatting characters.

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

package main

import (
"encoding/json"
"fmt"
"strings"
)

type RC string

func (r *RC) UnmarshalText(b []byte) error {
*r = RC(fmt.Sprintf("RC_%s", strings.ToUpper(string(b))))
return nil
}

type incident struct {
Regions map[RC]Region json:"regions"
}

type Region struct {
Facets []Facet json:"facets"
}

type Facet struct {
SOE string json:"sensitivity_override_explanation"
}

func main() {
r := 
{
"regions": {
"US": {
"facets": [{
"sensitivity_override_explanation": "43423432423\\r"
}]
}
}
}

i := new(incident)

if err := json.Unmarshal([]byte(r), &i); err != nil {
fmt.Println(err)
return
}

for k := range i.Regions {
fmt.Printf("key: %q\n", k)
}
}


### What did you expect to see?

key: "RC_US"

### What did you see instead?

key: "RC_US\": {\n\t\t\t"

closed time in 6 days

rkolp

issue commentgolang/go

Thanks @rkolp. It looks like @dsnet already did a bisect and a backport issue there, so let's consolidate both issues into that one. I realise this issue is older, so apologies for the mishap.

rkolp

comment created time in 6 days

issue commentmvdan/sh

Hm, this was a nasty bug. See the commit message for details. It will definitely ship with the upcoming release.

jumanjiman

comment created time in 6 days

push eventmvdan/sh

commit sha 837b6bf67e439821fc8eaf187626235260b3c65b

syntax: support disabling KeepPadding This was a big deal for shfmt's support for editorconfig files, since it relies on the ability to enable or disable options as per the currently applied editorconfig properties. There were two bugs here; the first, found by a user, was that enabling the option twice would crash. The second, which was silent and more dangerous, was that disabling the option did nothing. The latter could result in incorrect formatting options being used for some files. Fixes #530.

push time in 6 days

issue closedmvdan/sh

## Expected

Without printer options (such as -ci), shfmt should read .editorconfig for its options.

## Actual

• With a single file on the CLI, the program works as expected.
• When multiple files are passed on the CLI, the program panics.

I created a minimal reproducer at https://github.com/jumanjiman/reproducer-shfmt to demonstrate the issue.

💡 If I remove keep_padding from .editorconfig, it works as expected.

## Observed platforms

shfmt version: 3.0.2

The issue appears on:

• Mac (shfmt from Brew)
• Alpine (shfmt from apk add)

## Reproduce with Docker

To run the reproducer on Alpine with Docker:

docker build --rm -t repro .
docker run --rm -it repro


## Results

\$ ./test
[RUN] shfmt -version
v3.0.2

[INFO] One filepath with printer options is OK.
[RUN] shfmt -d -i 4 scripts/1
[RUN] shfmt -d -i 4 scripts/2
[RUN] shfmt -l -i 4 scripts/2

[INFO] One filepath without printer options is OK.
[RUN] shfmt -d scripts/1
[RUN] shfmt -d scripts/2
[RUN] shfmt -l scripts/2

[INFO] Two filepaths with printer options is OK.
[RUN] shfmt -d -i 4 scripts/1 scripts/2
[RUN] shfmt -d -ci scripts/1 scripts/2
[RUN] shfmt -l -ci scripts/1 scripts/2

[INFO] Two filepaths WITHOUT printer options is PANIC.
[RUN] shfmt -d scripts/1 scripts/2
panic: interface conversion: syntax.bufWriter is *syntax.colCounter, not *bufio.Writer

goroutine 1 [running]:
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/syntax/printer.go:58 +0xac
main.propsOptions(0x0, 0x0, 0xc0000d8800, 0xb, 0x10)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:257 +0x320
main.formatBytes(0xc0000d4000, 0x34, 0x600, 0x7ffeefbff9f3, 0x9, 0x8000, 0x8000)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:294 +0x74f
main.formatPath(0x7ffeefbff9f3, 0x9, 0x11dc000, 0x0, 0x0)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:285 +0x2a8
main.walk(0x7ffeefbff9f3, 0x9, 0xc0000b1f18)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:205 +0x84
main.main1(0xc000070058)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:173 +0x2fb
main.main()
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:63 +0x22

[ERROR] exit status 2: shfmt -d scripts/1 scripts/2

[RUN] shfmt -l scripts/1 scripts/2
panic: interface conversion: syntax.bufWriter is *syntax.colCounter, not *bufio.Writer

goroutine 1 [running]:
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/syntax/printer.go:58 +0xac
main.propsOptions(0x0, 0x0, 0xc0000d8800, 0xb, 0x10)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:257 +0x320
main.formatBytes(0xc0000d4000, 0x34, 0x600, 0x7ffeefbff9f3, 0x9, 0x8000, 0x8000)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:294 +0x74f
main.formatPath(0x7ffeefbff9f3, 0x9, 0x11dc000, 0x0, 0x0)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:285 +0x2a8
main.walk(0x7ffeefbff9f3, 0x9, 0xc0000b1f18)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:205 +0x84
main.main1(0xc000070058)
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:173 +0x2fb
main.main()
/private/tmp/shfmt-20200223-13327-c96be6/sh-3.0.2/cmd/shfmt/main.go:63 +0x22

[ERROR] exit status 2: shfmt -l scripts/1 scripts/2


closed time in 6 days

jumanjiman

issue commentmvdan/sh

This is fixed for the time being. All published binary and docker artifacts should be properly versioned from now on.

The only gaps left at the moment are go get mvdan.cc/sh/v3/cmd/shfmt@master, and git clone https://github.com/mvdan/sh && cd sh && go install ./cmd/shfmt. Both will get the devel version.

If we moved to the module build info, the first would start getting good version information, but not the second. So I don't think it's hugely important that we move to this fancier method today, as it's still not foolproof.

https://github.com/golang/go/issues/37475 seems to be getting traction now, so I think we might see some progress in the coming months.

oliv3r

comment created time in 6 days

issue commentmvdan/sh

Thanks for the very detailed bug report - this indeed sounds like a mistake somewhere.

jumanjiman

comment created time in 6 days

issue commentactions/runner

If that were true, wouldn't we be able to reproduce this locally? I'm completely unable to on Linux with Go 1.14.1, even after tens of thousands of runs with stress.

mvdan

comment created time in 6 days

push eventmvdan/sh

commit sha 75147f5680df459716ea8a1f458f81716eb138ce

interp: skip TestRunnerTerminalExec/Pseudo on all OSs Just got a failure on ubuntu-latest. The same error as Mac: --- FAIL: TestRunnerTerminalExec (0.00s) --- FAIL: TestRunnerTerminalExec/Pseudo (0.03s) ##[error] terminal_test.go:133: signal: hangup FAIL Updates #513.

push time in 6 days

push eventmvdan/sh

commit sha 343d511a6e94f3b38bedc15f57008a7200c3210a

interp: skip TestRunnerTerminalExec/Pseudo on all OSs Just got a failure on ubuntu-latest. The same error as Mac: --- FAIL: TestRunnerTerminalExec (0.00s) --- FAIL: TestRunnerTerminalExec/Pseudo (0.03s) ##[error] terminal_test.go:133: signal: hangup FAIL Updates #513.

push time in 6 days

issue commentactions/runner

Literally half an hour after filing this issue, I've just found the exact same failure on ubuntu-latest:

https://github.com/mvdan/sh/runs/539777043?check_suite_focus=true

So maybe this isn't related to Mac after all. It does seem to happen more on that platform, though. I'm unable to repro the failure locally on Linux, too

mvdan

comment created time in 6 days

push eventmvdan/sh

commit sha d08d7f42749793632c20aeaa4bcb1281bcb1722b

switch to checkout@v2, bump docker FROM versions checkout@v2 fetches only one commit by default, so fetch-depth is no longer necessary.

push time in 6 days

issue commentactions/runner

I haven't personally written a minimal reproduction program, but I imagine it would be possible given the pseudo-TTY Go library. It's a bit hard for us to come up with a reliable reproducer, given that we can't get a shell on a machine that sees the issue.

mvdan

comment created time in 6 days

issue commentactions/runner

I should clarify - there were another 4-5 failures, but those jobs were retried and succeeded, so I don't have the links anymore.

mvdan

comment created time in 6 days

issue openedactions/runner

Describe the bug

A Go test that makes use of pseudo-TTYs works fine on both Linux and Mac locally, and on Linux on Actions. However, on Mac on Actions, it fails pretty regularly, about a quarter of the time.

To Reproduce

1. Install Go 1.14.x
2. Clone https://github.com/mvdan/sh
3. Revert https://github.com/mvdan/sh/commit/ab43dfa8ae84bf3e4c10176a4bd4d837aa97ba9a to re-enable the flaky test
4. Run the test a bunch of times, via go test -run TestRunnerTerminalExec/Pseudo ./interp

You can hammer the test much harder by using stress. For example, to run 64 parallel processes until you hit a failure:

go test -vet=off -c -o test && stress -p=64 ./test -test.run TestRunnerTerminalExec/Pseudo


Expected behavior

No failures, ever. @thelapp was nice enough to run the stress line above, and he got zero failures even after 200k runs in over fifteen minutes. Likewise, I can't reprodue any issue on Linux.

## Runner Version and Platform

From a recent failed job on macos-latest today, Current runner version: '2.165.2'. Links below.

## What's not working?

The test fails with a weird signal: hangup error:

--- FAIL: TestRunnerTerminalExec (0.00s)
--- FAIL: TestRunnerTerminalExec/Pseudo (0.04s)
##[error]        terminal_test.go:129: signal: hangup
FAIL
FAIL	mvdan.cc/sh/v3/interp	3.529s


Below is a list of such failures in the past few weeks:

• https://github.com/mvdan/sh/pull/531/checks?check_run_id=538743036
• https://github.com/mvdan/sh/runs/507984876
• https://github.com/mvdan/sh/runs/462330729

I really don't know what else to try. Given that physical Mac machines of ours can't reproduce the issue, and that the error seems to come from a OS signal, I don't know how else to proceed but to raise a bug here.

/cc @myitcv @leitzler

created time in 6 days

PR merged chromedp/chromedp

Related to #440.

+3 -3

1 comment

1 changed file

rhcarvalho

pr closed time in 6 days

push eventchromedp/chromedp

commit sha ea717c104aa0249ddf6042072f310618c76c4be1

Document that BySearch matches by plain text too

push time in 6 days

pull request commentchromedp/chromedp

This is from https://chromedevtools.github.io/devtools-protocol/tot/DOM#method-performSearch:

Plain text or query selector or XPath search query.

So I agree - the wording seems to match what you've updated it to.

rhcarvalho

comment created time in 6 days

push eventmvdan/sh

commit sha 8c4fc3e75a3d274a0b569229619bd5f504a2b634

cmd/shfmt: build with latest git tag as version Instead of a fixed string as version for development releases, this injects the latest git tag as version during build and thus allows to identify the binary later. Fixes #519.

push time in 6 days

PR merged mvdan/sh

This tries to fix #519.

+2 -1

0 comment

1 changed file

spieth

pr closed time in 6 days

issue closedmvdan/sh

git describe --always --dirty is pretty amazing to help identify (un)tagged versions. It would be nice if shfmt would do this 'compile' time during release mode. I'm not familiar enough with go, but in C we tend to have a ./configure time generated version.h, which get the content at compile time. Doing a sed on the string would also work, but would make the file 'dirty', so better not that.

Especially when pulling the docker containers with the 'latest' tag, having identify-ability is quite useful, 'which version was used for this job, well version a.b.c-21-gdeadbeef is far more interesting then 'devel', even when the later statement is completely true :)

closed time in 6 days

oliv3r

push eventmvdan/sh

commit sha ab43dfa8ae84bf3e4c10176a4bd4d837aa97ba9a

interp: skip TestRunnerTerminalExec/Pseudo on Mac While we can't reproduce the failures on Mac laptops, GitHub's Mac machines fail this test pretty often. We can't have CI be that flaky. Updates #513.

push time in 6 days

issue commentmvdan/sh

Failed once again, this time on a PR: https://github.com/mvdan/sh/pull/531/checks?check_run_id=538743036

It's starting to affect our ability to rely on CI, so I'll start skipping the test. We should keep this issue open to investigate the true cause. My working theory is still that there's something different about Github's Mac machines.

mvdan

comment created time in 6 days

push eventopenbank/openbank

commit sha db86b72310503393801a98beac2f9c7e8a5a8e70

Fix enums, remove annexes Apply these PRs from gunk docgen: * https://github.com/gunk/gunk/pull/300 * https://github.com/gunk/gunk/pull/302 * https://github.com/gunk/gunk/pull/296 * https://github.com/gunk/gunk/pull/294

push time in 6 days

PR merged openbank/openbank

Reviewers

Apply these PRs from gunk docgen:

• https://github.com/gunk/gunk/pull/300
• https://github.com/gunk/gunk/pull/302
• https://github.com/gunk/gunk/pull/296
• https://github.com/gunk/gunk/pull/294
+2126 -1818

40 changed files

karel-3d

pr closed time in 6 days

issue commentgolang/go

I think I've hit the same bug in a long-running server with many thousands of goroutines. This early morning, the process just froze forever - even a goroutine that was simply logging some trivial stats every minute or so was not responding. One of the runnable goroutines is at runtime.resettimer, so that's a bit suspicious, and how I found this bug.

2020-03-26-hang-log.txt.gz

@mknyszek do you think it's the same bug, or should I file a new one?

I should note that this is a server that works fine with Go 1.13.x, but started hanging like this since we jumped to 1.14.x. I had been testing it with 1.14.x for weeks before, but we hadn't started running the long-lived staging servers with 1.14.x.until earlier this week.

mknyszek

comment created time in 7 days

pull request commentmvdan/gofumpt

Can you clarify what you mean by the wrong binary name?

It seems to me like the help text is supposed to tell you about the tool you're running, not about the way you happen to be calling it. This is the case with pretty much all official Go tools too:

func usage() {
fmt.Fprintf(os.Stderr, "usage: gofmt [flags] [path ...]\n")
flag.PrintDefaults()
}


I appreciate the patch, but I avoid os.Args[0] on purpose :)

I get that this tool is meant to be a replacement for an existing tool, and I do intend to keep it compatible with its flags and so on. But I don't think its usage text needs to be exactly the same as well.

theckman

comment created time in 8 days

pull request commentopenbank/openbank

@ghazninattarshah @nkristianto @jwangsadinata any last thoughts before we merge? if not, let's just merge tomorrow

karel-3d

comment created time in 9 days

issue commentgolang/go

Following the same idea, should go build ./foo/ fail?

perillo

comment created time in 9 days

issue commentgolang/go

Two more genuine use cases occurred to me, even in the face of lazy module loading being impemented:

• If one only wants non-test deps to be downloaded, go list ./... can download far less than go list -test ./...
• If one only wants deps required for a specific set of tags, go list -tags=xxx ./... might also avoid other unnecessary deps

And, like @jayconrod mentioned on slack, if one only needs the deps for a subset of the packages, like go list ./cmd/... if that's all that one needs to build offline later.

jayconrod

comment created time in 9 days

issue commentmvdan/garble

I've made the output collapsed by default, because it's huge.

Most of the output you're showing is from the standard library, like the runtime package. Have you read the README? For example:

The standard library is never garbled when compiled, since the source is always publicly available.

If you have ideas on how its obfuscation can be improved, issues are welcome. But a broad "leaks too much information" without more detail doesn't help me.

lu4p

comment created time in 9 days

push eventmvdan/gofumpt

README: closing parenthesis in grouped var example

push time in 9 days

PR merged mvdan/gofumpt

+1 -0

0 comment

1 changed file

kybin

pr closed time in 9 days

issue commentmvdan/garble

Thanks for filing this issue, and for the detailed information. It's likely the code has a bug somewhere; I'll try to investigate later this week.

lu4p

comment created time in 9 days

issue commentmvdan/garble

Thanks for filing this issue! I'll take a look when I can.

suntong

comment created time in 9 days

issue commentgolang/go

It also wasn't clear to me that go mod download was synonymous with go mod download all. Maybe go help mod download could be made a bit more explicit.

jayconrod

comment created time in 10 days

PR closed golang/go

The current implementation of the glob in Match(pattern, name) function is evaluating the pattern syntax in a lazy way, this can lead to some mistakes, due to a call with an invalid pattern can end up without match or returning ErrBadPattern error depending on the input.

This change forces the pattern syntax validation, avoiding inconsistent error return depending on the input, this change shouldn't break the compatibility even without the correct error handling but can imply a performance penalty when the input is empty.

Fixes #28614

Change-Id: I055a3b707b93f8ddda87551effb90f5e767a4117

+118 -34

2 changed files

jmartin82

pr closed time in 10 days

pull request commentgolang/go

CL was abandoned at the request of the author.

jmartin82

comment created time in 10 days

issue commentgolang/go

I wonder if we should wait for more progress on https://github.com/golang/go/issues/35950, given that one could think of tzdata as an opt-in asset file. Either way, I agree that a solution to this issue would be very useful.

ianlancetaylor

comment created time in 10 days

issue commentkeybase/client

For completeness, both xdg-open and open do the right thing when run from a terminal. Starting keybase-gui manually from said terminal after doing a systemctl --user stop keybase.gui doesn't fix anything, either. I normally start the GUI via the desktop launcher.

mvdan

comment created time in 10 days

issue openedkeybase/client

Clicking on https links used to work fine. I run keybase-gui on Arch, version 5.3.0, with Electron 8.1.1.

Now, clicking on https links does nothing. If I do journalctl --user -f and check the logs, I can see messages like:

Mar 23 15:02:01 carbon electron[75612]: could not find program for '68', check your configuration


This message is very confusing. Is it coming from electron, or from keybase? What does 68 even stand for, here? What configuration should I be checking?

Any hints are very much appreciated. I have no ideahow to investigate this any further.

Perhaps related to https://github.com/keybase/client/issues/22782, though I can't find any similar details other than the overall problem description.

created time in 10 days

issue closedmvdan/sh

closed time in 11 days

wangfuli217

issue commentmvdan/sh

wangfuli217

comment created time in 11 days

push eventgunk/gunk

commit sha 27300022660c858c8588def989091fe792e7c8a9

docgen: use fmt.Errorf(%w) after switch to go1.13 We use go 1.13 everywhere, so removing one TODO.

push time in 11 days

PR merged gunk/gunk

Reviewers
+1 -3

0 comment

1 changed file

karel-3d

pr closed time in 11 days

push eventmvdan/github-actions-golang

add a link to the command docs Thanks to Francis Lavoie for pointing this out in #6.

push time in 11 days

issue commentmvdan/github-actions-golang

Ah, my bad! It wasn't documented when I last looked :)

francislavoie

comment created time in 11 days

issue closedgolang/go

I'm from Casbin community. My user sent me a PR to add glob pattern matching support: https://github.com/casbin/casbin/pull/164. The glob pattern matching is done by a 3rd-party library: https://github.com/gobwas/glob . However, I don't want to bring another dependency. I think glob matching is very commonly-needed feature, so can we build it into Go built-in library like regex?

I noticed there's a filepath.Glob() here: https://golang.org/pkg/path/filepath/#Glob but it's only for matching file names. Can we broaden it to a general-purpose string matching? Thanks.

closed time in 11 days

hsluoyz

issue commentgolang/go

If I want to have a general-purpose glob matching that have the same behavior for all OSes, should I use path.Match() instead?

Yes. It doesn't seem like there's anything to do here, so I'm closing this issue.

hsluoyz

comment created time in 11 days

issue commentgolang/go

Thanks for the report, @smasher164. Let's take this to https://github.com/mvdan/gofumpt/issues/36, since the bug doesn't concern gofmt.

smasher164

comment created time in 12 days

IssuesEvent

issue commentmvdan/gofumpt

@smasher164 raises a possible bug at https://github.com/golang/go/issues/34582#issuecomment-599244959. Reopening for now.

mvdan

comment created time in 12 days

push eventmvdan/nowt

commit sha cd0e36df724f83c5d04af987431c9372d46ab190

push time in 12 days

push eventmvdan/nowt

commit sha 8b937e64aaff8eb55e2b09af714bf598bfc71649

go-mod-size: first tool

push time in 12 days

create barnchmvdan/nowt

created branch time in 12 days

created repositorymvdan/nowt

Nothing extraordinary here

created time in 12 days

push eventmvdan/gofumpt

commit sha 4bdc928868e60b5ac1a986e3565f24c579868f80

add spaces to comments less aggressively If any of the lines in a comment group look like shouldn't be touched, don't touch any of the group at all. See the added test case and comments. Fixes #43.

push time in 12 days

issue closedmvdan/gofumpt

Right now, we decide if one single comment line is definitely not plain English given its first character, and then decide not to add a space prefix to it.

If that's the case, the same should apply to its whole "paragraph", i.e. the consecutive comment lines (before or after) which aren't empty.

closed time in 12 days

mvdan

push eventmvdan/gofumpt

commit sha 7590cc6ce5206387b1a41901486df87281401123

obey named or commented std imports in the top group If the developer chose to put them there, obey that. Start documenting this code better too, as it's getting tricky. Fixes #45.

push time in 12 days

issue closedmvdan/gofumpt

The following import group

import (
_ "expvar" // metrics
"net/http"
"net/http/pprof"

"github.com/gorilla/mux"
)


is now being reordered as

import (
"net/http"
"net/http/pprof"

_ "expvar" // metrics

"github.com/gorilla/mux"
)


breaking our linter

./pkg/metrics/metrics.go: Import groups are not in the proper order: ["Std" "Std" "Third party"]


closed time in 12 days

ksegun

issue openeddominikh/go-tools

The simplest form might be to find pairs of build tags which have separate files, where some number of top-level declarations are exact duplicates. If the two tags share any file, suggest to have the declaration there instead.

Follow-up thoughts:

1. This shouldn't be an exhaustive search of all possible combinations. For example, you could first filter by which build tags have top-level declarations with matching hashes (e.g. given the stringified syntax tree without comments).

2. You might or might not want to include tiny pieces of code, like single-line constants. Depends on the false positive ratio one finds in the wild.

3. This could be considered for cases where the build tags don't share any files at all, too. Though I expect most packages would at least have one file that doesn't have any build tags, like doc.go or api.go.

4. Long-term, this should work for any set of build tags, not just pairs. I think starting with pairs is probably the simplest.

created time in 13 days

issue commentgolang/go

@dfurtado json encoding should be deterministic. Ranging over a map has an unspecified order, so there isn't a stable order we can use by default. So the encoder falls back to the equivalent of sort.Strings.

lavalamp

comment created time in 13 days

push eventmvdan/github-actions-golang

commit sha fc93475119a5c7176d98e6374aa6003155777884

push time in 13 days

more