profile
viewpoint
Daniel Martí mvdan Sheffield, United Kingdom https://mvdan.cc I work on stuff in Go.

govim/govim 483

govim is a Go development plugin for Vim8, written in Go

mvdan/github-actions-golang 360

GitHub Actions as CI for Go

gunk/gunk 287

Modern frontend and syntax for Protocol Buffers

mvdan/garble 179

Obfuscate Go builds

mvdan/fdroidcl 97

F-Droid desktop client

mvdan/accesspoint 75

Manage wireless access points in Android (abandoned)

mvdan/dockexec 33

Run Go tests inside a Docker image

mvdan/bitw 25

Minimalist BitWarden client

mvdan/benchinit 24

Benchmark the init cost of Go packages

mvdan/git-picked 19

List merged and cherry-picked branches

push eventmvdan/sh

Daniel Martí

commit sha cc3ea662c91b9cdcc621f40b773a398d155bdb94

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. The exec method also has real FDs 0/1/2, versus the native version's, whose global FDs 0/1/2 belong go 'go test'. We remove the IsTerm interface too, as it was initially added mainly for testing, and it's no longer strictly needed. We can re-add it in the future as part of the API, if we think it's useful. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. While at it, switch from x/crypto to x/term for IsTerminal. It's the new home for this API, and the module is far smaller. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Thanks to Larry once again for helping get the pty tests working on Mac. Fixes #508, again.

view details

push time in 9 hours

issue closedmvdan/sh

interp: [[ -t n ]] test looks at the wrong file handles

The -t code looks at file handle n directly, which will be global for the entire process, but if n == 0, 1, or 2 it should look at Runner.stdin, stdout, or stderr, which will be specific to each Runner.

closed time in 9 hours

theclapp

PR merged mvdan/sh

interp: test tty detection properly

See the commit message.

+149 -47

8 comments

7 changed files

mvdan

pr closed time in 9 hours

pull request commentmvdan/sh

interp: test tty detection properly

OK, so here are the major takeaways:

  1. The Runner and Exec tests must both use the same threading model. Before, Runner ran synchronously via Run, and Exec ran asynchronously via Start. Now Runner spawns a goroutine to mimick Cmd.Start.

  2. The other major problem related to waiting. Initially, we didn't close either end of a pty/pipe. This resulted in ReadAll hanging forever. You fixed this initially with a single Read call, but I didn't find that answer very satisfactory. What if it's a partial read? How do we know we're done reading?

Then, I resorted to having the master/writer close the connection. This worked most of the time, but it was racy - if the reader hadn't had time to read all of the bytes we want, it could end up with EOF, or an even worse error like "pipe already closed".

The root of the problem here is that neither side knows when we're "done". Each test case writes a different number of bytes, so we don't know either. I fixed this by forcing every test case to expect exactly one line of output. This way, the reader can stop as soon as it's read one line, and the writer can then close.

Phew :) @theclapp if you're happy with the latest patch, I'll merge. It seems all green, and there are no skips.

mvdan

comment created time in 12 hours

push eventmvdan/sh

Daniel Martí

commit sha 411baf27e2e622a4f0bdfcb9e9ebf54c3e9fa496

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. The exec method also has real FDs 0/1/2, versus the native version's, whose global FDs 0/1/2 belong go 'go test'. We remove the IsTerm interface too, as it was initially added mainly for testing, and it's no longer strictly needed. We can re-add it in the future as part of the API, if we think it's useful. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. While at it, switch from x/crypto to x/term for IsTerminal. It's the new home for this API, and the module is far smaller. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Thanks to Larry once again for helping get the pty tests working on Mac. Fixes #508, again.

view details

push time in 13 hours

pull request commentmvdan/sh

interp: test tty detection properly

I think I'm wrapping my mind around the problem. I have a potential solution I'm trying out, which should be portable and non-racy.

mvdan

comment created time in 13 hours

push eventmvdan/sh

Daniel Martí

commit sha 75cbf870bf4cc9fc73e548656fa92a2c5e9e4a18

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. The exec method also has real FDs 0/1/2, versus the native version's, whose global FDs 0/1/2 belong go 'go test'. We remove the IsTerm interface too, as it was initially added mainly for testing, and it's no longer strictly needed. We can re-add it in the future as part of the API, if we think it's useful. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. While at it, switch from x/crypto to x/term for IsTerminal. It's the new home for this API, and the module is far smaller. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Thanks to Larry once again for helping get the pty tests working on Mac. Fixes #508, again.

view details

push time in 13 hours

pull request commentmvdan/sh

interp: test tty detection properly

Ah, the Exec one still fails on Darwin, but you weren't reenabling that one. Hm.

mvdan

comment created time in 15 hours

push eventmvdan/sh

Daniel Martí

commit sha 3daacfa16187e7cbc9ea7e52cbbe1c861fefa013

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. The exec method also has real FDs 0/1/2, versus the native version's, whose global FDs 0/1/2 belong go 'go test'. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. While at it, switch from x/crypto to x/term for IsTerminal. It's the new home for this API, and the module is far smaller. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Thanks to Larry once again for helping get the pty tests working on Mac. Fixes #508, again.

view details

push time in 15 hours

pull request commentmvdan/sh

interp: test tty detection properly

Did you by any chance generate that patch via git diff -w? It doesn't apply properly :) It was simple enough, so I just pieced it back together by hand. In the future, I think git diff without any options is the only sure way to produce a working patch.

In any case - thanks so much for the help! I do give up on Mac affairs pretty quickly, simply because it's a nightmare to test on it. I think I was able to simplify your version a bit, and it still passes on Linux. I'll push and see if it still passes on Mac.

mvdan

comment created time in 15 hours

issue commentgolang/go

encoding/json: document change in Number behavior for 1.14

Thanks all for the input - I'll have a CL ready for the changelog doc later today.

marwan-at-work

comment created time in 17 hours

issue commentgolang/go

encoding/json: json.Number breaks behavior in Go 1.14

I agree with @bcmills - the bare minimum here is to clarify this in the release notes.

There is a tradeoff here - technically following the compatibility policy, but breaking half the Go programs out there, would clearly be a bad idea. However, I don't think that's the case here, and like @myitcv showed, there are workarounds. Your code isn't guaranteed to work exactly the same between Go versions, especially if you don't stick to documented behavior.

marwan-at-work

comment created time in 17 hours

issue commentgolang/go

encoding/json: json.Number breaks behavior in Go 1.14

If the only problem is empty strings, would my workaround above be enough?

We have a rule that we start testing new release only when they reach rc1

Sure, though rc1 came out two weeks ago, and the final release is scheduled just a few days away. We can try to have a fix before then, but this late into the cycle, it might have to wait until 1.14.1.

marwan-at-work

comment created time in 20 hours

pull request commentmvdan/sh

interp: test tty detection properly

@theclapp requesting a review, if you have a few minutes :)

mvdan

comment created time in 21 hours

push eventmvdan/sh

Daniel Martí

commit sha 0c40cf60053ddeea1ba4c9dee73ec469d9471619

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Fixes #508, again.

view details

push time in 21 hours

push eventmvdan/sh

Daniel Martí

commit sha cf43d74500301295720f32e700bef2d84ff2185a

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Fixes #508, again.

view details

push time in 21 hours

push eventmvdan/sh

Daniel Martí

commit sha 91677b3e553b74268c4b8358bcbd88b25726b7f5

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Fixes #508, again.

view details

push time in 21 hours

push eventmvdan/sh

Daniel Martí

commit sha fb235534ccf1fc3330a0472f950f3d78f0844dac

interp: test tty detection properly This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain. The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method. We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there. Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp. Fixes #508, again.

view details

push time in 21 hours

create barnchmvdan/sh

branch : 508-terminal-tests

created branch time in a day

PR opened mvdan/sh

interp: test tty detection properly

This includes two table-driven tests to detect whether FDs 0-3 are terminals. One uses the Go API with StdIO, without spawning a separate process, and another uses os/exec to run the interpreter in a separate process via TestMain.

The cases are very similar, except that with pure Go we can fake a buffer to be a terminal via the IsTerm method.

We also add github.com/creack/pty, so that we can test a pseudoterminal in both cases. This isn't supported on Windows, so don't run the tests there.

Finally, a few changes to the 'test -t' logic needed to be made to properly support all of the combinations above. The mistakes in the old code were correctly spotted by Larry Clapp.

Fixes #508, again.

+138 -30

0 comment

5 changed files

pr created time in a day

issue commentgolang/go

encoding/json: json.Number breaks behavior in Go 1.14

Also CC @breml since he wrote the original patch.

marwan-at-work

comment created time in a day

issue commentgolang/go

encoding/json: Unmarshal into struct field or a map does not validate invalid json.Number

Please continue the discussion in #37308. If you have a sample of the real scenario that broke for you, please add it there.

brenol

comment created time in a day

issue commentgolang/go

encoding/json: json.Number breaks behavior in Go 1.14

This should definitely be in the release notes :) thanks for bringing it up.

It's unfortunate that noone has noticed until now - it's true that rc1 was very late, but it has been over two months since 1.14beta1.

Can you provide a reproducer? I'm particularly interested in what kind of input you're seeing in the wild.

This issue is not a duplicate of #34472, but they are related. json.Number is documented to only accept JSON numbers when decoding, but in practice it seems to accept any string. #14702 was about invalid numbers (which caused this issue), and #34472 is about accepting quoted numbers even when the ,string option isn't used.

Thinking outloud, I wonder if we could amend https://golang.org/cl/195045 to keep accepting empty strings, if that's where the vast majority of the breakage comes from. I think not accepting "foobar" as a json.Number was a good change.

marwan-at-work

comment created time in a day

issue commentgolang/go

Can't recover from go: unknown environment setting GO111MODULE=autoGO111MODULE=auto

@jedebarbieri this issue is closed, and is not the place to ask questions. See https://golang.org/wiki/Questions.

dylankb

comment created time in 2 days

issue commentmvdan/sh

Allow for automatic aligning of command-continuing backslashes

Friendly ping @Varriount.

Also CC @dcsobral, who didn't reply on the original thread.

The simplest change we could do is enforce these to be aligned all the time, like we do with comments.

The alternative would be to have -kp keep their padding, and not enforce any alignment ourselves. This would be harder, as it would require us to add the \ characters to the syntax tree somewhere, with position information.

The third alternative is to keep things as is and do nothing.

Varriount

comment created time in 2 days

push eventmvdan/sh

Daniel Martí

commit sha b86cb96eff0dc2cf78c4d40847446e7459a43bcf

interp: support aliases in the 'type' builtin It uses bash's weirdly mismatching `' quotes, just so that the test is consistent. We might replace this with regexp in the future. Fixes #479.

view details

push time in 2 days

issue closedmvdan/sh

interp: 'type' doesn't work with the new 'alias' code

type shows words as builtins and functions and so forth, but doesn't work with aliases.

gosh
$ type type
type is a shell builtin
$ function f { date ; }
$ type f
f is a function
$ alias f2=date
$ type f2
type: f2: not found
$

closed time in 2 days

theclapp

issue commentmvdan/sh

Tabs to indent, spaces to align

@oliv3r just to clarify, are you OK with the above? If not, I don't see another way we could tackle this feature request.

oliv3r

comment created time in 2 days

issue commentgolang/go

Values of millisecond lost when using Mongo Driver to read time and Unmarshalling int into time.Time

Why are you filing this issue here? You should probably file it with the mongo driver you're using.

Abhaykumar1

comment created time in 2 days

issue closedmvdan/sh

echo "${.sh.version}" breaks shfmt

When I try to format my korn shell scripts with shfmt, then this line triggers an error.

echo "${.sh.version}"
$ shfmt -w -i 4 version 
version.sh:3:9: invalid parameter name

Please don't present errors for valid ksh string interpolation syntax.

closed time in 2 days

mcandre

issue commentmvdan/sh

echo "${.sh.version}" breaks shfmt

We support POSIX Shell, Bash, and mksh. As far as I know, that's not valid syntax in any of them.

You already raised an issue about ksh support in https://github.com/mvdan/sh/issues/161, but I got no answer, just like with multiple other of your issues. I'm not going to work on ksh support unless there's a good reason for it, so I'm closing this for now. Feel free to reply later and I might re-open.

mcandre

comment created time in 2 days

issue commentmvdan/sh

shfmt: Write to files atomically instead of in place

Permissions are easy to propagate.

Can you clarify how? For example, os.Chown doesn't work at all on Windows.

nhooyr

comment created time in 2 days

issue commentmvdan/sh

shfmt: Write to files atomically instead of in place

@ArturKlauser I don't think atomic file writing packages for Go go that far. I agree that permission and ownership must be kept. I don't think the others are as important or common, but it would be ideal to keep them too.

Whatever we come up with, remember that it must be portable. And I'd strongly prefer to not add an option for this.

Any ideas, @nhooyr?

nhooyr

comment created time in 2 days

push eventmvdan/sh

Daniel Martí

commit sha 54c096194eecc24044dac2dc84127fadb2719a2e

interp: split interp.go into runner.go and api.go Now we have two files at ~600 and ~700 lines, instead of one at ~1.3k lines. The separating line is roughly public API versus runner internals. We can also merge doc.go into api.go, as it fits that file better.

view details

push time in 2 days

issue commentgolang/go

proposal: cmd/go: add GOMODCACHE

My preference is for the first option because the second option is a bit clunky.

I agree - as long as the migration of the old sumdb dirs is smooth.

Do we want to specify a relationship between the GOMODCACHE variable, and what we could set GOPROXY to be to proxy from the module cache?

It seems to me like that could be decided as a separate issue, as long as we don't close that door in this proposal.

mvdan

comment created time in 2 days

push eventmvdan/sh

cclerget

commit sha 23ddfbcf8b296fa13b530eae79e83bd92f5b7fe9

allow naked exports of readonly variables That is, allow: readonly foo=bar export foo

view details

push time in 3 days

PR merged mvdan/sh

Fix issue with readonly and naked export.

Doing the following should not return a readonly variable error

readonly foo=bar
export foo
+11 -1

0 comment

2 changed files

cclerget

pr closed time in 3 days

issue commentmvdan/sh

cmd/shfmt: official docker images

For what it's worth, once v3.0.2 is out, the -alpine image will be available. It's already available for master as latest-alpine.

mmlb

comment created time in 3 days

issue closedmvdan/sh

interp: test flake with -race on mac

https://github.com/mvdan/sh/runs/417718558

$ go test -short -race -count=1 ./...
ok  	mvdan.cc/sh/v3/cmd/gosh	1.275s
ok  	mvdan.cc/sh/v3/cmd/shfmt	10.646s
ok  	mvdan.cc/sh/v3/expand	1.185s
?   	mvdan.cc/sh/v3/fileutil	[no test files]
--- FAIL: TestRunnerRun (0.19s)
    --- FAIL: TestRunnerRun/779 (0.02s)
##[error]        interp_test.go:2559: wrong output in "sed 's/o/e/g' <(echo foo bar)":
            want: "fee bar\n"
            got:  "open /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/sh-interp-3747852017: no such file or directory"
FAIL
FAIL	mvdan.cc/sh/v3/interp	2.472s

A real bug, or a weird mac-related problem? Probably the former. Sadly, I don't have a mac machine to test with.

closed time in 3 days

mvdan

issue commentmvdan/sh

interp: test flake with -race on mac

Wow, I can't believe that lazy attempt worked :) I'll reopen if CI spots it again.

mvdan

comment created time in 3 days

issue commentcuelang/cue

Ability to add helpful error messages for humans

I'd add the errors label, but I assume only people with write access to the repo can do that.

mvdan

comment created time in 3 days

issue openedcuelang/cue

Ability to add helpful error messages for humans

Issues like https://github.com/cuelang/cue/issues/52 and https://github.com/cuelang/cue/issues/171 would be a step in the right direction, so that automatic errors are more helpful.

However, there's generally quite a bit of context that cue will never be able to infer on its own. For example:

$ cat test.cue
import "list"

needs_foo: bool
foo: [...string]

if needs_foo {
	foo: list.MinItems(1)
}
$ cat test.yaml
needs_foo: true
foo: []
$ cue vet -c test.yaml test.cue
foo: invalid operation [] &! list.MinItems (1) (mismatched types list and list):
    ./test.cue:4:6
    ./test.cue:7:7
    ./test.yaml:2:7

It would be far better if cue vet gave a useful error message, like foo can't be an empty list when needs_foo is true. This is particularly important when the consumer of the errors isn't familiar with cue, or when they don't have easy access to the cue files which are vetting their yaml or json.

One can imagine other scenarios where a human-friendly error message would be far better than a generated one. For example, in the context of Kubernetes, high-level errors can be explained simply, such as Deployment "foo" refers to unknown ConfigMap "bar".

Such human-friendly errors would be optional, and ideally replace generated errors. My initial thought was along the lines of:

foo: list.MinItems(1) // foo can't be an empty list when needs_foo is true

Here's a variant from @rogpeppe:

foo: list.MinItems(1) | error("foo can't be an empty list when needs_foo is true")

@mpvl also had some thoughts on Slack, but I'll let him reply here as that seems better than copy-pasting all of his messages.

created time in 3 days

IssuesEvent

issue commentmvdan/sh

interp: [[ -t n ]] test looks at the wrong file handles

Let's reopen until we add proper tests with https://github.com/creack/pty.

theclapp

comment created time in 3 days

CommitCommentEvent

issue commentmvdan/sh

interp: test flake with -race on mac

So, we do something a bit weird. When we create a FIFO on disk, the writer deletes the file as soon as it's done writing. That seems to work well on Linux, and I haven't encountered a single flake. It seems to be flaky on Mac - maybe our code is buggy, or maybe Mac has different guarantees on deleting FIFOs with opened readers.

I've also pushed a couple of commits that might help (see above); if anyone can re-test on a Mac machine, that would be helpful. Otherwise, I'll keep an eye on CI to see if it fails again. CI appears to get about one failure per week due to this flake on Mac.

mvdan

comment created time in 3 days

push eventmvdan/sh

Daniel Martí

commit sha 510bc041abd58b55c926b3b65a012e28ce32cce9

interp: replace syscall.Mkfifo with x/sys/unix.Mkfifo In an attempt to make progress for #493. The syscall package is deprecated, and Mac changes pretty regularly, so there's a chance that the errors were caused by obsolete syscall code.

view details

Daniel Martí

commit sha f1c23200944c0bf4d0cff5999d122e742be031bc

update deps

view details

push time in 3 days

issue commentmvdan/sh

interp: exit statuses do not reflect being killed by a signal

Writing a proper test for this turned out to be a bit of a nightmare. But the change itself was easy.

theclapp

comment created time in 3 days

push eventmvdan/sh

Daniel Martí

commit sha 2295fef3ff9b1e8885b5fda6717c585bb0480821

interp: adjust exit status codes for signals To follow Bash's principle: The return value of a simple command is its exit status, or 128+n if the command is terminated by signal n. It's unclear if POSIX Shell requires this behavior, but the behavior isn't incompatible with POSIX, as it only defines status codes up to 127. Fixes #500.

view details

push time in 3 days

issue closedmvdan/sh

interp: exit statuses do not reflect being killed by a signal

Bash manpage says

The return value of a simple command is its exit status, or 128+n if the command is terminated by signal n.

Bash:

bash-3.2$ sleep 120
# ... kill the sleep from another window ...
Terminated: 15
bash-3.2$ echo $?
143

gosh:

$ sleep 120
# ... kill the sleep from another window ...
$ echo $?
255

closed time in 3 days

theclapp

push eventmvdan/sh

Daniel Martí

commit sha f480de2db3b6f38b0a2369302e859ebee8294525

interp: don't use global file descriptors in test -t Instead, support IsTerm and Fd methods on the interface values. The latter is supported by *os.File, and the former can be implemented like we do in our test. In the future, access to the global file descriptors will be possible when the "single process mode" is used, as described in #171. For now, we never go that route. Fixes #508.

view details

push time in 3 days

issue closedmvdan/sh

interp: [[ -t n ]] test looks at the wrong file handles

The -t code looks at file handle n directly, which will be global for the entire process, but if n == 0, 1, or 2 it should look at Runner.stdin, stdout, or stderr, which will be specific to each Runner.

closed time in 3 days

theclapp

issue commentmvdan/sh

interp: add a single process mode

Another use case: [[ -t 0 ]] tests if stdin is a terminal. This should only query the real file descriptor 0 if we are in "single process" mode.

mvdan

comment created time in 3 days

push eventmvdan/sh

Daniel Martí

commit sha 3c5828bd79cacb971d8ce11f36ed171fd96bb436

interp: rework the logic from the last commit First, replace the keepParams boolean field with a narrower one that has a better name. We now only modify it when inside a sourced script. Second, dont do an unnecessary 'r.Params = r.Params' assignment. Third, carefully document each step of the source builtin, since it's starting to get a bit hard to follow.

view details

push time in 3 days

issue openedgolang/go

cmd/go: add a way to obtain a package's build ID via list -json

The toolchain computes build IDs of packages from their build input - which I believe includes a package's source code, plus the build IDs of all their dependencies.

This can be seen if one runs go build -x, as compiler invocations contain -buildid $BUILD_ID. Similarly, a package's output file contains such build ID as well, so we can obtain it via these two commands:

$ go list -f {{.Export}} -export os
/home/mvdan/tip/pkg/linux_amd64/os.a
$ go tool buildid $(go list -f {{.Export}} -export os)
WCQPVNh2igTNOQyg4V2a/aDZysAb7fmq4QDTG8hDz

This is rather unfortunate for tools that need to look at thousands of packages at a time, though. One could use the build ID to see if a package hasn't had any changes, and thus avoid extra work. Even if we have thousands of packages, we can load them all at once with one single go list -json call - however, we will need one extra exec call to go tool buildid per package, whose cost can add up quickly.

I propose that we add a new field to go list -json to expose said build ID. For example, BuildID string. It would be fine if populating this field required a flag like -export.

I also realise that build IDs are an internal toolchain detail, which should only matter to compiler and toolchain authors. However, there are plenty of third-party tools out there which are designed to be closely coupled with the toolchain or compiler. For example, tools designed to run via go build -toolexec=mytool can use build IDs to avoid work, or to produce deterministic output based on the input. This is exactly the kind of thing I do for a code obfuscator, only with lots of go tool buildid calls.

I also am not asking for the build ID format to be documented or stabilized. A tool shouldn't interpret the string as more than a hash if it wants to remain compatible with past and future Go versions. Or, if it really wants to parse the hash, it should only do so with very specific versions of the Go tool.

/cc @dominikh

created time in 3 days

issue commentgolang/go

encoding/json: json.Number accepts quoted values by default

@mvrhov there has been no change yet. If there was, this issue would not be open with the NeedsDecision label.

@puellanivis it's indeed a tough choice. If we can't come to an agreement on changing it (which seems to be the case), then it's probably too late or painful to change it. I can have a look at a documentation change for 1.15, to at least have the documentation reflect the internal behavior that too many people have been relying on already.

vearutop

comment created time in 3 days

push eventmvdan/sh

cclerget

commit sha facd8ba38a5a073f3d58deb713bda7cb618001ec

interp: propagate parameter changes from sourced scripts Currently sourced scripts can't change interpreter parameters; the following script should display 'arg1 arg2;: set -- cat > /tmp/source <<EOF set -- arg1 arg2 EOF source /tmp/source echo $@

view details

push time in 3 days

PR merged mvdan/sh

Fix an issue not propagating interpreter parameters changes from sourced scripts.

Currently sourced scripts can't changes interpreter parameters, the following script should display arg1 arg2:

set --
cat > /tmp/source <<EOF
set -- arg1 arg2
EOF
source /tmp/source
echo $@
+74 -3

0 comment

3 changed files

cclerget

pr closed time in 3 days

pull request commentmvdan/sh

Pre commit hook

I'm not against the concept of pre-commit hooks :) But I am against the concept of having to add a yaml file that doesn't really do anything beyond saying "hey, I support this pre-commit framework!". The tiny amount of data it's actually defining could very well be inferred, or declared by the actual user of pre-commit.

Note that related tools (https://github.com/psf/black, https://github.com/pycqa/flake8) all provide them.

One of those tools has a dozen "dotfiles" in the root of the repository - I'm not sure that's a good example to strive towards :)

casperdcl

comment created time in 4 days

push eventmvdan/sh

Daniel Martí

commit sha 6c51941c3ce288af4c818c05861420d82dad9cf9

syntax: rename FunctionNewLine to FunctionNextLine On second thought, this name is more consistent with previous options like BinaryNextLine. We also don't introduce ambiguity with the casing, since NewLine could easily be mixed up with Newline. While at it, update the variable names, and make them a bit shorter.

view details

push time in 4 days

push eventmvdan/sh

Daniel Martí

commit sha 76c819a5c2a3e80afe4fdcde7a6cb7af465b3296

syntax: rename FunctionNewLine to FunctionNextLine On second thought, this name is more consistent with previous options like BinaryNextLine. We also don't introduce ambiguity with the casing, since NewLine could easily be mixed up with Newline. While at it, update the variable names, and make them a bit shorter.

view details

push time in 4 days

issue commentmvdan/sh

syntax: configurable formatting of opening braces

@MorganAntonsson's work is now merged! Please test it now in master, as 3.1.0 will probably release at some point in March. Once released, it's going to be impossible for us to make any breaking changes to the feature.

abhijithda

comment created time in 4 days

push eventmvdan/sh

Morgan Antonsson

commit sha c3b7a40ab2d75cd8c7ae89df038bf7458e6cfe5d

cmd/shfmt: add a -fn printer option This option allows placing a function's opening brace on a separate line, which brings one's style a bit closer to K&R. It can also be useful to quickly find or jump to function definitions. The same option is exposed in the syntax package. Fixes #229.

view details

push time in 4 days

issue closedmvdan/sh

syntax: configurable formatting of opening braces

Thanks for the great tool. I have started using this tool with the VS Code. I would like to provide one feedback w.r.t formatting functions (which would at least help couple of us).

The opening brace of the function definition should be in next line.

Currently it's as below...

function fun() {
}

Suggested way...

function fun()
{
}

Refer examples here: https://www.shellscript.sh/functions.html

One advantage of following the suggested format is, in the terminal editors like (vim), you can jump between functions by pressing [ and ] character twice. This is very important feature for specially scripts - as once you are done developing the script on your desktop, you would copy/run it on a system where there may not be graphical interfaces or IDEs.

closed time in 4 days

abhijithda

PR merged mvdan/sh

Add two new printer options: -knl and -bnl

Hi,

I have added two new printer options for a less compact style.

-knl (keyword new line) will put keywords like "then" and "do" on a new line.

Example:

if foo
then
    bar
fi

-bnl (brace new line) will put braces on new lines.

Example:

foo()
{
    bar
}

It would be great if you would accept the patch. Let me know if there is anything I need to change.

Regards, Morgan

+55 -13

6 comments

3 changed files

MorganAntonsson

pr closed time in 4 days

issue commentmvdan/sh

interp: [[ -t n ]] test looks at the wrong file handles

That's a good catch - patches welcome. Any other instances of file descriptor handling should be checked for the edge case of stdin/stdout/stderr too.

theclapp

comment created time in 4 days

issue commentgolang/go

x/mobile: build failing when using go modules

@themartorana x/mobile is a repository and a module that you can use today. It's not shipped with Go releases, nor does it follow its release schedule. If a new version of gomobile is out, you can use it with Go 1.13.8, just like you can use it with 1.14rc1.

mirza-s

comment created time in 4 days

pull request commentmvdan/sh

Pre commit hook

This was brought up before; see https://github.com/mvdan/sh/pull/465. Please read that thread to get up to speed.

I'm still not convinced that this is necessary. Running the tool is already very simple, via either a binary or a Docker image. It also seems to me like if someone really wants to explicitly support third party tools or software, that could be done in separate git repos. For example, people have written vscode and vim plugins, but they don't live here because I don't maintain them.

Please understand that I appreciate the contribution - I just don't quite get what a small yaml file at the root of the repository is really adding here.

casperdcl

comment created time in 4 days

push eventmvdan/garble

Daniel Martí

commit sha 1ce53104405d2a53c125f01dadd5c3a9ec59643a

don't garble exported struct fields They might reasonably affect the behavior of the code, such as when encoding/json is used without tags.

view details

Daniel Martí

commit sha 4d5ad43f10c89f12b578b954fa81c4e3a2422833

allow garble to test itself With this patch, 'go install && garble test' works.

view details

push time in 4 days

push eventmvdan/garble

Daniel Martí

commit sha b10cce34f84a3beaef885215b4e4507664b2b136

parse boolean flags differently from string flags This is important, because "-std -foo" and "-buildid -foo" are entirely different cases. The first is equivalent to "-std=true -foo" since the flag is boolean, but the second is equivalent to "-buildid=-foo" since the flag isn't boolean. We can keep track of which of the flags we're interested in are boolean, which isn't much extra work. Also add unit tests; the build ID is a hash, so it's very hard to write an end-to-end test that reliably has an ID starting with a dash.

view details

push time in 5 days

push eventmvdan/garble

Daniel Martí

commit sha ce0137fa6a71702fcac984cb87f6460f7b61d7c4

don't break TestMain funcs Important for 'garble test', if a package uses one.

view details

push time in 5 days

delete branch mvdan/garble

delete branch : test

delete time in 5 days

issue commentgolang/go

modules: changes/corruption in module cache not detected

I think you're after go mod verify:

$ go help mod | grep verify
	verify      verify dependencies have expected content

I believe that other commands like go get and go build will only check a module's contents when downloading the module for the first time. Doing that again any time a build happens, or any time a module's dependencies are queried, could be pretty expensive and hurt incremental builds.

This is explained a bit in go help modules:

No matter the source of the modules, the go command checks downloads against known checksums, to detect unexpected changes in the content of any specific module version from one day to the next.

Also:

The reason I raise this is that I on rare occasions have seen empty cache folders. I'm not able to give further details about when/how this happens, but for me, it happens only on MacOS. In most situations, this will be visible (break the build), but not always.

If that is your problem, I think we should investigate it. Until the bug is investigated, go mod verify is a manual alternative.

CC @bcmills @jayconrod

bep

comment created time in 6 days

push eventmvdan/sh

Daniel Martí

commit sha 7a8db93945ce7eb1f1cd2a029d1039ad4e21f828

interp: expand a bit on exit vs lastExit The difference is a bit subtle.

view details

push time in 6 days

push eventmvdan/sh

Daniel Martí

commit sha b71ab34ebe8f39c4b2840bb2a5194d5415285b59

interp: handle the last exit status code better In some edge cases, such as commands with no arguments or background commands, we didn't properly reset it to 0. Doing that breaks expansions like "$?"; to properly handle all cases, we need to keep track of the current and last status codes separately, at least while we execute a statement. As a bonus, we can deduplicate a few places where we set r.exit = 0. Fixes #506.

view details

push time in 6 days

issue closedmvdan/sh

interp: exit status ($?) not being reset correctly in some edge cases

I think gosh is only resetting Runner.exit when it runs an external command. At very least, it's not setting it when it should.

bash:

% false ; echo $?
1
% false ; n=6 ; echo $?
0

The specific n=6 command is irrelevant, I just want a command that a) succeeds and b) wouldn't "obviously" set $? but still should.

gosh:

$ false ; echo $?
1
$ false ; n=6 ; echo $?
1

This might be fallout from #499. I can take a look at it later.

closed time in 6 days

theclapp

startedevanw/esbuild

started time in 6 days

issue commentgolang/go

cmd/go: document module-mode behavior of multi-element GOPATHs

What is the reason for this choice?

The choice was arbitrary, and it doesn't really matter anymore :) I dropped that configuration over a year ago when I fully switched all my projects to modules.

mvdan

comment created time in 7 days

issue closedgolang/go

Question: Why pointers' zero value is not the zero value of what they points to?

This is a question to get some information before thinking about a possible proposal regarding zero values.

Could you please let me know or point me to an authoritative document where I can read about the reasons why the zero value of a pointer is nil instead of a pointer to the zero value of what it points to?

Is it because of performance or memory? Simplicity in the runtime?

Example:

type Pet struct {
	name string
	kind string
}
type Person struct {
	name    string
	age     int
	friends []Person
	pet     Pet
}

// Current Go
var person1 Person // { name: "", age: 0, friends: [], pet: <nil> }
var person2 *Person // <nil>

// Assuming the zero value of a pointer is a pointer to the zero value of what it points to, then:
var person1 Person // { name: "", age: 0, friends:[], pet: &{ name: "", kind: "" } }
var person2 *Person // &{ name: "", age: 0, friends:[], pet: &{ name: "", kind: "" } }

Thanks!

closed time in 7 days

alvaroloes

issue commentgolang/go

Question: Why pointers' zero value is not the zero value of what they points to?

See the very top of the issue template:

For questions please use one of our forums: https://github.com/golang/go/wiki/Questions

alvaroloes

comment created time in 7 days

issue commentmvdan/sh

Exit status ($?) not being reset correctly

I'll also take this one from you, as it's far from trivial. We need to split up "current status code" from "last status code".

theclapp

comment created time in 7 days

push eventmvdan/sh

Daniel Martí

commit sha dedd5982ba2a0c13c79055cad9cd36ee4d4cf17e

CI: stop updating the info of the old alpine hub repo The alpine images are now tags on the main docker repo, instead of their own separate repo. See #504.

view details

push time in 7 days

push eventmvdan/sh

Olliver Schinagl

commit sha 78debaf8cfd6341ee04962381525690eba0afbfa

push alpine container to the proper repository We want the alpine container to show up as a tag, not as its own repo. Fixes #504 Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>

view details

push time in 7 days

PR merged mvdan/sh

Push alpine container to the proper repository

We want the alpine container to show up as a tag, not as its own repo.

Signed-off-by: Olliver Schinagl oliver@schinagl.nl

Closes #504

+1 -1

0 comment

1 changed file

oliv3r

pr closed time in 7 days

issue closedmvdan/sh

CI: alpine images are getting pushed to a different repo

They're going to https://hub.docker.com/r/mvdan/shfmt-alpine, e.g. mvdan/shfmt-alpine:3.0.1, instead of https://hub.docker.com/r/mvdan/shfmt, e.g.mvdan/shfmt:3.0.1-alpine`.

If the worry is about latest, then we can simply publish the latest-alpine tag as well.

/cc @ArturKlauser @oliv3r

closed time in 7 days

mvdan

issue commentmvdan/sh

Exit status ($?) not being reset correctly

I don't think your PR is to blame. It doesn't seem like we ever had a test case for this.

theclapp

comment created time in 7 days

push eventmvdan/sh

Olliver Schinagl

commit sha 34860fdfcb1cb9e5dc6a12a67ee67e0a865db3ef

remove old release-docker.sh script With this repo having moved to auto-builds, we no longer need this script. Signed-off-by: Olliver Schinagl <oliver@schinagl.nl>

view details

push time in 8 days

PR merged mvdan/sh

release: Also build and push alpine containers

We currently have two containers to choose from during the build stage, an alpine container and a scratch container.

The scratch container is there for size, where we do not want or need anything other then just the raw binary.

The alpine container exists for when more is needed, such as a shell or apk to add packages.

CI pipelines need the later, due to the requirement of needing a shell among things. To cater this need, also build the alpine based container and push it with the -alpine tag, much like other multi-target docker containers do.

Whilst here, also clean-up the script a little.

+0 -14

5 comments

1 changed file

oliv3r

pr closed time in 8 days

pull request commentmvdan/sh

release: Also build and push alpine containers

You're right. It's been pushing to https://hub.docker.com/r/mvdan/shfmt-alpine/tags, which is an entirely new repo I didn't intend to create. Let's move that to a separate issue: https://github.com/mvdan/sh/issues/504

oliv3r

comment created time in 8 days

issue openedmvdan/sh

CI: alpine images are getting pushed to a different repo

They're going to https://hub.docker.com/r/mvdan/shfmt-alpine, e.g. mvdan/shfmt-alpine:3.0.1, instead of https://hub.docker.com/r/mvdan/shfmt, e.g.mvdan/shfmt:3.0.1-alpine`.

If the worry is about latest, then we can simply publish the latest-alpine tag as well.

/cc @ArturKlauser @oliv3r

created time in 8 days

issue openedcuelang/cuelang.org

netlify: add redirects for /issue

To mirror golang.org. /issues?/(.*) redirects to github.com/cuelang/cue/issues/<id>.

This is particularly useful for Gerrit, since issue IDs don't work when reviewing, and it's easy to type cuelang.org/issue/X on the browser.

I'll be taking a look at this soon.

created time in 8 days

push eventmvdan/sh

Larry Clapp

commit sha 745956839951013634291f1ca4982590b17f1dfc

interp: make "type" builtin respect the current $PATH Fixes #502.

view details

push time in 8 days

PR merged mvdan/sh

interp: make "type" builtin respect the current $PATH

Fixes #502.

+14 -9

2 comments

2 changed files

theclapp

pr closed time in 8 days

issue closedmvdan/sh

interp: "type" does not respect $PATH if it's set within the shell

actual:

% gosh
$ PATH=/
$ type ping
ping is /sbin/ping

expected — something more like:

% PATH=/ ./gosh
$ type ping
type: ping: not found

Edit: It's actually a problem with "type"; see below.

closed time in 8 days

theclapp

Pull request review commentmvdan/sh

interp: make "type" builtin respect the current $PATH

 import ( 	"mvdan.cc/sh/v3/syntax" ) +// Some program which should be in $PATH.  Needs to run before runTests is+// initialized, below.

explain why - I assume it's because we want to use it in "expected output" string literals

theclapp

comment created time in 9 days

Pull request review commentmvdan/sh

interp: make "type" builtin respect the current $PATH

 func TestMain(m *testing.M) { 	} 	os.Setenv("GOSH_PROG", prog) -	os.Setenv("LANGUAGE", "en_US.UTF8")-	os.Setenv("LC_ALL", "en_US.UTF8")+	os.Setenv("LANGUAGE", "en_US.UTF-8")+	os.Setenv("LC_ALL", "en_US.UTF-8")

I agree this is a good change. Not sure why the old version worked. I think it was a typo of mine.

theclapp

comment created time in 9 days

issue commentmvdan/sh

"type" does not respect $PATH if it's set within the shell

Ah yeah, we probably didn't implement this edge case properly. Patches welcome, otherwise I'll have a look eventually.

theclapp

comment created time in 9 days

pull request commentmvdan/sh

release: Also build and push alpine containers

You're right! Complete brain fart on my part, for some reason I thought this was still being used as part of CI/CD. Yes, we should delete it. @oliv3r you can do it as part of this PR, or if not, I'll do it in master.

oliv3r

comment created time in 9 days

push eventgosheffield/admin

Daniel Martí

commit sha f717e3981ed2f918a09a667069d65bbc9aa45a4c

meeting_notes: add today's notes

view details

push time in 9 days

issue closedchromedp/chromedp

How to Get the current URL as a String

<!--- This issue tracker is mainly for bugs and feature requests. Before asking a question, search online and try to investigate on your own. -->

What versions are you running?

<pre> $ go list -m github.com/chromedp/chromedp v0.5.3 $ google-chrome --version 80.0.3987.87 $ go version 13.7 </pre>

What did you do? Include clear steps.

What did you expect to see?

how to Get the current URL as a String with chromedp. thank you.

What did you see instead?

closed time in 9 days

longjin1991
more