Alex Brainman alexbrainman

odbc driver written in go

Windows printing

Windows SSPI

Small Windows GUI app written in Go

Windows jobs

Windows performance counters (MANY THINGS ARE BROKEN)

Delve is a debugger for the Go programming language.

issue commentgolang/go

My particular case is the Direct2D and DirectWrite APIs, although I've seen the need elsewhere as well.

Which API are you talking about?

Another possibility, which would require only a single code change in function runtime·asmstdcall in runtime/sys_windows_amd64.s and no visible API changes: make use of the currently-unused r2 result by always copying the X0 register into it on return, much like is currently done for the first 4 potential floating point arguments.

That sounds reasonable. Is X0 register documented by Microsoft to receive returned value on amd64?

This value is used on 32-bit systems, but I don't think the float problem exists there and no change is needed in runtime/sys_windows_386.s.

Fair enough.

Alex

richardwilkes

comment created time in a day

issue commentgolang/go

Based on the MoveFileEx documentation and various threads on Windows Q&A sites and mailing list, I think the most likely explanation for this issue is that another process (probably antivirus or search indexer) is opening files in the module directory before it's completely extracted.

That sounds reasonable. Especially given that you have small example program that reproduces the problem.

I googled for MoveFileEx ERROR_ACCESS_DENIED and I found

https://habr.com/ru/post/450700/

While it is not exactly the same issue, but similar. It talks about

The DeleteFile function marks a file for deletion on close. Therefore, the file deletion does not occur until the last handle to the file is closed. Subsequent calls to CreateFile to open the file fail with ERROR_ACCESS_DENIED.

But the article gave me an idea. The author used Process Monitor

to monitor all syscalls that are happening to his file. We could do the same.

@alfmos you could use Process Monitor to log all accesses to files with path starting with d:\go\pkg\mod\golang.org\x\tools@v0.0.0 or similar. This will tell you which processes (Go tools and other programs) access yor file in question and how. Especially the bit where the failure occurs. Please try it.

I don't think we can fix this,

Why not? The problem is that you use MoveFileEx, and it does not work for your use case. Surely there are different ways to design your cache storage process - the way that does not require moving files.

Alex

kiwionly

comment created time in a day

issue commentgolang/go

WSASendto calls WSASendTo (which seems a little nuts to me, but okay...),

https://golang.org/cl/3136042

These were the days where syscall package was free for all to do as we pleased.

What if we were to directly invoke the syscall from WSASendto with the unsafe.Pointer that came from to.sockaddr(), instead of converting it through an intermediate *RawSockaddrAny?

I will try and report it back here.

Actually, WSASendTo (with the big 'T') is definitely an unsafe API, ...

You have to complain to Microsoft

https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsasendto

Even better. You should complain to people who designed Berkeley sockets in the first place.

Can we just remove WSASendTo?

We can do anything you want. But we are using the function ourselves in net package. And other people might be using it too. So we would have to replace it with something that call real WSASendTo anyway.

Alex

mdempsky

comment created time in 5 days

issue commentgolang/go

@kpfaulkner please see, if you can test this change

https://golang.org/cl/219657

I'm pretty sure my original fix for this issue rather than the whack a mole as was requested by interlocuteurs is preferable.

I agree. My approach did not work.

Alex

kpfaulkner

comment created time in 5 days

issue commentgolang/go

@ihgann

I don't know if there is an error here - I am not a crypto expert.

But I would like to try and reproduce your error here first. You said nothing about your server and how you generated your certificate. Is your server is written in Go? Can I see your code? What are the commands to generate and use your certificate?

Thank you.

Alex

ihgann

comment created time in 5 days

issue commentgolang/go

I don't know for certain, but judging by a quick Google search, it appears to be possible in Ruby, so I assume one can do that in C-based languages.

You are correct. I was wrong. os.Chdir does work with UNC paths.

Alex

bk2204

comment created time in 13 days

issue commentgolang/go

why change make.bat? Can we make all these changes in cmd/dist instead?

Forget it. That was silly idea. GOROOT_BOOTSTRAP is needed to build cmd/dist.

Alex

FiloSottile

comment created time in 13 days

issue commentgolang/go

### What did you expect to see?

The same results as calling GetFinalPathNameByHandle: a UNC path.

I did not try it, but, I suspect, UNC paths wouldn't work in some situations. For example, can you pass UNC path to os.Chdir?

Alex

bk2204

comment created time in 13 days

issue commentgolang/go

I am seeing exactly this issue(A timeout was reached (90000ms) while waiting for the Docker Engine service to connect​) for docker on our machines now and then, and the timeout in our case is 90s.

I had some free time today. And I tried running this service golang.org/x/sys/windows/svc/example on my Windows 10 64 bit computer. I rebooted it 5 times, and service started without any problems every single time.

I can try and debug this issue again, but I need to be able to reproduce it on my computer. Does anyone knows of a Go service that I can build myself to reproduce this issue? I don't know what Docker Engine service is and how to build it.

Thank you.

Alex

kneden

comment created time in 25 days

issue commentgolang/go

On Unix we automatically use $(go env GOROOT) as GOROOT_BOOTSTRAP (see #14339). We should do that on Windows as well, where it's harder to set env vars, and for consistency. Make sure that there are checks in place to prevent $GOROOT_BOOTSTRAP == \$GOROOT.

@FiloSottile why change make.bat? Can we make all these changes in cmd/dist instead?

We had complicated make.bat before. But then we created cmd/dist and moved all complicated logic there. It is OK to do simple things, like setting an environment variable or calling a program, in make.bat. But what you are asking, is not simple.

This https://go-review.googlesource.com/c/go/+/208197/5 is the latest version of this fix. I did not verify it yet. And it is incomplete, because it will not compare GOROOT and GOROOT_BOOTSTRAP values properly - we need case-insensitive comparison here. Are you comfortable with CL 208197 changes? Will you be OK to debug / support this make.bat when it breaks? Because I am not comfortable.

And, if we change cmd/dist, it will cover all GOOS platforms with single code (mostly).

Alex

FiloSottile

comment created time in 25 days

issue commentgolang/go

With the sample I gave, this will work just fine on big-endian platforms as the GUID stored in that LDAP attribute is in little-endian.

You are correct here. I was wrong with my example.

I am still against making any of golang.org/x/sys/windows available on non Windows. But I won't stop others.

Alex

dcormier

comment created time in a month

issue commentgolang/go

CC @jstarks @alexbrainman @zx2c4 @networkimprov in case they have any insights into how to debug this.

I don't have any good ideas.

I am not familiar with that code. And I don't have free time to learn the code to debug it.

Alex

kiwionly

comment created time in a month

issue commentgolang/go

I agree https://github.com/golang/go/issues/35006#issuecomment-546573751 might help.

You can also

set CGO_ENABLED=0

and that will build Go without any gcc installed. You won't be able to do Cgo, if you do that.

Thank you.

Alex

lovettchris

comment created time in a month

issue commentgolang/go

I merged this fix into rclone however @Ajaja reported in rclone/rclone#3888 that the Enter key no longer works.

Sorry for submitting completely broken version.

Yes. The fix https://go-review.googlesource.com/c/crypto/+/215417 is out for review.

Alex

ncw

comment created time in a month

issue commentgolang/go

But the gist of it is:

Good enough for me.

obejctGUID := guid.FromWindowsArray(guidValue)


ToWindowsArray

// ToWindowsArray returns an array of 16 bytes representing the GUID in Windows
// encoding.
func (g GUID) ToWindowsArray() [16]byte {
return g.toArray(binary.LittleEndian)
}


https://github.com/microsoft/go-winio/blob/6c72808b55902eae4c5943626030429ff20f3b63/pkg/guid/guid.go#L120-L124

uses binary.LittleEndian. So, even if we make golang.org/x/sys/windows.GUID available on all platforms supported by Go, ToWindowsArray function will be broken everywhere where binary.LittleEndian is not default encoding.

So you still need to talk to the owners of github.com/Microsoft/go-winio/pkg/guid package, if you want to use it on non-Windows.

Alex

dcormier

comment created time in a month

issue commentgolang/go

Right now, syscall.StartProcess requires that exactly three files be passed to the child process. If we start a process on Windows with bInheritHandles set to false, what will standard input, output, and error be?

From STARTUPINFOW doco

hStdInput

If dwFlags specifies STARTF_USESTDHANDLES, this member is the standard input handle for the process. If STARTF_USESTDHANDLES is not specified, the default for standard input is the keyboard buffer.

and

STARTF_USESTDHANDLES 0x00000100 The hStdInput, hStdOutput, and hStdError members contain additional information.If this flag is specified when calling one of the process creation functions, the handles must be inheritable and the function's bInheritHandles parameter must be set to TRUE.

So, if CreateProcess bInheritHandles is false, then STARTF_USESTDHANDLES cannot be used, and lpStartupInfo.hStdInput cannot be used, so child process input will be keyboard.

Same for stdout and stderr.

But, we could, probably, use new STARTUPINFOEX to overcome this problem.

https://devblogs.microsoft.com/oldnewthing/?p=8873

Alex

zhaoya881010

comment created time in a month

issue commentgolang/go

@johmue thank you for filling this issue.

Thank you.

Alex

johmue

comment created time in a month

issue commentgolang/go

... If we fail silently for some bad reason, how bad is it actually? And if the power notifier is borked, how much else from the power stack is borked? ...

You assume that if PowerRegisterSuspendResumeNotification returns error than Windows power stack is broken. But I am not convinced that this is true. It seems that way on Docker, but we haven't had any reports of PowerRegisterSuspendResumeNotification error on non Docker. So it seems to be working fine on non Docker, and we should care about any PowerRegisterSuspendResumeNotification failure on Windows.

And things might change in time. For example, Microsoft might decide to implement PowerRegisterSuspendResumeNotification in Docker.

Also maybe we could somehow determine, if we running inside Docker, and only ignore errors, if in Docker.

https://stackoverflow.com/questions/43002803/detect-if-process-executes-inside-a-windows-container

And I was under impression that you cared about Windows programs clocks, because of WireGuard. How can you be certain about clocks, if you ignore PowerRegisterSuspendResumeNotification errors? You know it is broken in Docker. But you still want to be sure about your Windows clients. Aren't you?

Alex

kzettlmeier

comment created time in a month

issue commentgolang/go

Certainly if at all possible we should be doing this in the standard library rather than requiring the program to use some interface to list the descriptors that should be closed.

I believe normal Go program does not leave any parent process file handles opened in child process. Except 3 handles of child stdin, stdout and stderr.

CreateProcess is always true

Yes. CreateProcess bInheritHandles is always set to true by Go. But that is not enough to make a process handle inherited by a child. From

If this parameter is TRUE, each inheritable handle in the calling process is inherited by the new process.

So only inheritable handles are inherited by child process. And Go does not have any inheritable handles. Except the 3 I described above.

@zhaoya881010 do you actually have some problem to show us? Or you just read source code and think that the code is wrong?

Alex

zhaoya881010

comment created time in a month

issue commentgolang/go

There are GUIDs in there that I need to handle. The existing windows.GUID struct appears to be the correct type to use.

Can you, please, show small code that demonstrates your point. How exactly will you use the struct? What will you program do?

Alex

dcormier

comment created time in a month

issue commentgolang/go

Looks like error 87 is ERROR_INVALID_PARAMETERS, which doesn't make much sense to me.

I think this errors means - this code is not supported on Docker - in this context. There are only so many Windows error messages available, so, as a programmer, you have to pick best exiting number that fits.

And ERROR_INVALID_PARAMETERS fits much better than ERROR_FILE_NOT_FOUND. We are looking for ERROR_FILE_NOT_FOUND returned from PowerRegisterSuspendResumeNotification at this moment to detect Docker environment.

I think we should add ERROR_INVALID_PARAMETERS to the list of errors we are looking for there.

Or maybe just ignore any errors returned from PowerRegisterSuspendResumeNotification, as @zx2c4 originally suggested. It does not feel right ignoring all errors on non-Docker.

Unrelated, but maybe we should run one of our builders in Windows Docker container. I believe, Windows Docker containers are available on GCP. Maybe having Docker builder will pick errors like this. @bradfitz what do you think?

Alex

PS: @ianlancetaylor I normally google for windows error numbers or something. And that search page brings some links to pages with all Windows errors. Once there, you can search the page by error number or name or even error message.

kzettlmeier

comment created time in a month

PR opened go-delve/delve

Specifically, make sure that both DebugActiveProcess and WaitForDebugEvent Windows APIs are executed on the same thread.

Otherwise WaitForDebugEvent fails with ERROR_INVALID_HANDLE as per its documentation

https://docs.microsoft.com/en-us/windows/win32/api/debugapi/nf-debugapi-waitfordebugevent

'... Only the thread that created the process being debugged can call WaitForDebugEvent. ...'

Fixes #1825

+6 -3

0 comment

1 changed file

pr created time in a month

push eventalexbrainman/delve

commit sha a782abee904a8b73fa1935531250a572d9cc9e5a

Update vendored version of peterh/liner (#620) As suggested by Ederson de Souza. https://groups.google.com/forum/?utm_medium=email#!msg/delve-dev/W_SfC3i9tf0/MY4QWaZOCgAJ

commit sha f2c1789c643f31bea11c337230a316996e98ed13

Changed TestIssue426 to work with go 1.8 (#627) go1.8 changed the way anonymous struct names are generated for DWARF.

commit sha e4c7df14698fdb2fcec4e56200f4006a6b91bcc2

Fix prologue detection on testvariables2.go's main.main (#625)

cmd: Support space separate flags for build-flags (#619) Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>

commit sha 4d8daeb1a884c475b0b63483e3e15727f7af1c90

proc: implement detach on windows (#615)

commit sha 5d78c04e629869c2ab2a1f63b6c2d856b99507b1

dlv: bugfix: Allow quoting in build flags argument (#639) Allows quoted substrings in build-flags flag. This fixes a build problem on windows where the default build flags must contain a space. Fixes #634 and #638

commit sha fb8388ab2e50ca28ef9a83c73c329f8d0b6234d9

proc/variables: missing return statement on error (#643) Fixes #642

commit sha 8d5826202070566defb6df6ea95b93b8c96b4d24

proc: bugfix: Next and normal calls to deferred function When a deferred function is called directly Next and StepOut should not step into it. Fixes #582

commit sha 7c49d4968d84cc017b4e254dc307784f80108132

proc: Implement Step using Continue Instead of repeatedly calling StepInstruction set breakpoints to the destination of CALL instructions (or on the CALL instructions themselves for indirect CALLs), then call Continue. Calls to unexported runtime functions are skipped. Reduces the number of code paths managing inferior state from 3 to 2 (StepInstruction, Continue). Fixes #561

commit sha 9cbe76883617e7db782d4ba1011afb704b0cc303

proc: Renamed temp breakpoints to internal breakpoints

commit sha 46803551b888c017e22bef94d6bf223a9eb704a3

CI: add Go 1.7 to Travis (#649)

commit sha f6e8fb37a40312a363baec4cdde5a1ee66debf59

proc: changed windows backend to deal with simultaneous breakpoints (#598) * proc: changed windows backend to deal with simultaneous breakpoints * bugfix: forgot to add windowsPrologue3 to the prologues list in e4c7df1 * Tolerate errors returned by Stacktrace in TestStacktraceGoroutine. * bugfix: proc: propagate debug events we don't cause back to the target process Fixes: #594 * proc: fixed TestStepConcurrentPtr Implementation of nextInProgress was wrong.

commit sha 6e882c50fa85aba34e4204e1016bc36ab09a617f

debugger/locations: prioritize exact matches of function names (#651) If the location specification matches the name of a function exactly return that function as a match event if the expression matches other functions as well. Without this some functions, like math/rand.Intn are unmatchable.

commit sha 54d3eab63a3e879a8c5253bd8808c4b744ffbfe8

prettyprint: Print type of the elements of arrays of interface type (#591)

commit sha 0f4b5150c370467c6478bc73b7ed64b92d94af56

proc, terminal: stepout command Command to step out of the currently executing function. Implements #358

commit sha 35bc789b3b37caabaf5af5e2f68f777e322aa44d

config, terminal, command: Add substitute-path parameter to configuration file. (#640) Allows to rewrite a source path stored in program's debug information, if the sources were moved to a different place between compilation and debugging.

commit sha f62bf8d1e3d6cd0ebe9e327fe9bbba31356fa1e9

proc: Names of concrete types of interfaces parsing their runtime._type Generate names of the concrete types stored inside interface variables by fully parsing their runtime._type instead of simply using the str field. This allows delve to read the contents of an interface variable when the program imports multiple packages that have the same name. It also allows delve to correctly interpret some complex anonymous types. Fixes #455

commit sha 4064d6acc06f89e9a2c336d01323b11e7a132d78

Flag to set working directory (#650) * proc: Add wd to Launch This change adds the wd arg which specify working directory of the program. Fixes #295 * service/debugger: Add Wd field to debugger.Config This change adds the Wd field which specify working directory of the program launched by debugger. Fixes #295 * service: Add Wd to service.Config This change adds the Wd field which specify working directory of the program debugger will launch. Fixes #295 * cmd/dlv: Add Wd flag This change adds Wd flag which specify working directory of the program which launched by debugger. Fixes #295 * only set the Linux working directory if it is set, stub out param in darwin and windows * set working directory for Windows https://godoc.org/golang.org/x/sys/windows#CreateProcess https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx * Windows workingDir must be an *uint16 * attempt to chdir on darwin via @yuntan * proc/exec_darwin.c: fix working directory for darwin * Add tests to check if working directory works. * Fix darwin implementation of fork/exec, which paniced if child fork returned. * cmd, service: rename Wd to WorkingDir

proc/variables: extend sign of read negative integers. (#657) Since we store all signed integers as int64 the sign bit should be extended, otherwise we read negative integers as their 2-complement value.

commit sha ee6d2734c92f2b8ca42fbb34f6e815acff218702

tests: fix TestBuild on macOS, windows

push time in a month

create barnchalexbrainman/delve

created branch time in a month

issue openedgo-delve/delve

1. What version of Delve are you using (dlv version)?

1. What version of Go are you using? (go version)?

go version go1.13.6 windows/amd64

1. What operating system and processor architecture are you using?

Windows 7

1. What did you do?

I run

go test -v -run=AttachDetach github.com/go-delve/delve/pkg/proc


command

1. What did you expect to see?

I expected test to pass.

1. What did you see instead?

The test fails, like this

=== RUN TestAttachDetach
--- FAIL: TestAttachDetach (5.58s)
proc_test.go:111: failed assertion at proc_test.go:2845: Attach - The handle is invalid.
FAIL
FAIL github.com/go-delve/delve/pkg/proc 6.184s
FAIL


The The handle is invalid. (ERROR_INVALID_HANDLE) is returned from WaitForDebugEvent Windows API.

Only the thread that created the process being debugged can call WaitForDebugEven ...

What, I think, that means, is that DebugActiveProcess and WaitForDebugEvent APIs should be called by the same thread. But that is not the case at this moment.

The error does not happen every single time I run the test, but it happens often enough.

Alex

created time in a month

issue commentgolang/go

Maybe we just need to hold the suspendLock for longer (though I don't have a theory for why this would be). What happens if you move the unlock(&suspendLock) in preemptM to the very bottom of the function?

I changed the code, like this

diff --git a/src/runtime/os_windows.go b/src/runtime/os_windows.go
index 91e147fca9..9166aeb323 100644
--- a/src/runtime/os_windows.go
+++ b/src/runtime/os_windows.go
@@ -1201,8 +1201,6 @@ func preemptM(mp *m) {
// GetThreadContext actually blocks until it's suspended.

-       unlock(&suspendLock)
-
// Does it want a preemption and is it safe to preempt?
gp := gFromTLS(mp)
if wantAsyncPreempt(gp) && isAsyncSafePoint(gp, c.ip(), c.sp(), c.lr())
{
@@ -1231,6 +1229,8 @@ func preemptM(mp *m) {

+
+       unlock(&suspendLock)
}

// osPreemptExtEnter is called before entering external code that may


And I managed to run make.bat once to successful completion. But then it hung as before, when I run make.bat second time. This time, it is compile.exe that hung. Here is the stack trace from windbg

0:009> !uniqstack

.  0  Id: 2c08.3140 Suspend: 1 Teb: 000007fffffde000 Unfrozen
Start: compile+0x67ef0 (0000000000467ef0)
Priority: 0  Priority class: 32  Affinity: f
000000000022fd98 000007fefcd610ac ntdll!ZwWaitForSingleObject+0xa
000000000022fda0 0000000000467f5e KERNELBASE!WaitForSingleObjectEx+0x9c
000000000022fe40 000000000160d320 compile+0x67f5e
000000000022fe48 0000000000000001 compile+0x120d320
000000000022fe50 0000000000000000 0x1

.  1  Id: 2c08.243c Suspend: 1 Teb: 000007fffffdc000 Unfrozen
Priority: 0  Priority class: 32  Affinity: f
0000000003cdfcb8 000007fefcd61430 ntdll!ZwWaitForMultipleObjects+0xa
0000000003cdfcc0 0000000076d806c0 KERNELBASE!GetCurrentProcess+0x40
0000000003cdfdc0 000000007482eed4 kernel32!WaitForMultipleObjects+0xb0
0000000003cdfe50 000000007480ad07 tmmon64+0xaeed4
0000000003cdff00 000000007480aeae tmmon64+0x8ad07
0000000003cdff30 0000000076d859cd tmmon64+0x8aeae
0000000003cdff60 0000000076fba561 kernel32!BaseThreadInitThunk+0xd
0000000003cdff90 0000000000000000 ntdll!RtlUserThreadStart+0x21

.  2  Id: 2c08.20ec Suspend: 1 Teb: 000007fffffda000 Unfrozen
Start: ntdll!RtlDestroyHandleTable+0x270 (0000000076faf6f0)
Priority: 0  Priority class: 32  Affinity: f
0000000003fefcc8 0000000076faed15 ntdll!ZwWaitForWorkViaWorkerFactory+0xa
0000000003fefcd0 0000000076d859cd ntdll!RtlValidateHeap+0x155
0000000003feff60 0000000076fba561 kernel32!BaseThreadInitThunk+0xd
0000000003feff90 0000000000000000 ntdll!RtlUserThreadStart+0x21

.  3  Id: 2c08.2ecc Suspend: 1 Teb: 000007fffffd8000 Unfrozen
Start: ntdll!TpIsTimerSet+0x8b0 (0000000076faa280)
Priority: 0  Priority class: 32  Affinity: f
00000000041efcb8 0000000076faa3c7 ntdll!ZwWaitForMultipleObjects+0xa
00000000041efcc0 0000000076d859cd ntdll!TpIsTimerSet+0x9f7
00000000041eff60 0000000076fba561 kernel32!BaseThreadInitThunk+0xd
00000000041eff90 0000000000000000 ntdll!RtlUserThreadStart+0x21

.  4  Id: 2c08.2cf4 Suspend: 1 Teb: 000007fffffd4000 Unfrozen
Start: compile+0x68330 (0000000000468330)
Priority: 0  Priority class: 32  Affinity: f
00000000299def98 0000000076fd8f58 ntdll!ZwWaitForSingleObject+0xa
00000000299defa0 0000000076fd8e54 ntdll!RtlDeNormalizeProcessParams+0x5a8
00000000299df050 0000000076fde0b5 ntdll!RtlDeNormalizeProcessParams+0x4a4
00000000299df080 0000000076fdddd8 ntdll!RtlAllocateHeap+0x455
00000000299df260 000007fefcd61635 ntdll!RtlAllocateHeap+0x178
00000000299df370 000007fefa36bf7d KERNELBASE!LocalAlloc+0x75
00000000299df3e0 000007fefa36c187 TmUmEvt64!TmmonDestoryAddonObject+0x63d
00000000299df430 000007fefa36bbf1 TmUmEvt64!TmmonDestoryAddonObject+0x847
00000000299df4b0 0000000074862cd3 TmUmEvt64!TmmonDestoryAddonObject+0x2b1
00000000299df580 00000000748629f4 tmmon64+0xe2cd3
00000000299df640 00000000747b3de4 tmmon64+0xe29f4
00000000299df6b0 0000000076d72a4a tmmon64+0x33de4
00000000299df780 0000000000467f5e kernel32!GetThreadContext+0xa
00000000299df7b0 ffffffffffffffff compile+0x67f5e
00000000299df7b8 0000000000000000 0xffffffffffffffff

.  5  Id: 2c08.ea4 Suspend: 1 Teb: 000007fffffae000 Unfrozen
Start: compile+0x68330 (0000000000468330)
Priority: 0  Priority class: 32  Affinity: f
0000000029bdfb08 000007fefcd610ac ntdll!ZwWaitForSingleObject+0xa
0000000029bdfb10 0000000000467f5e KERNELBASE!WaitForSingleObjectEx+0x9c
0000000029bdfbb0 000000c00002c980 compile+0x67f5e
0000000029bdfbb8 0000000000000000 0xc00002c980

.  6  Id: 2c08.558 Suspend: 2 Teb: 000007fffffac000 Unfrozen
Start: compile+0x68330 (0000000000468330)
Priority: 0  Priority class: 32  Affinity: f
0000000029ddf530 0000000076ffcf71 ntdll!RtlUnlockHeap+0x6a8
0000000029ddf5c0 0000000076fdddd8 ntdll!TpAlpcRegisterCompletionList+0xaf91
0000000029ddf7a0 000007fefcd61635 ntdll!RtlAllocateHeap+0x178
0000000029ddf8b0 000007fefa36bf7d KERNELBASE!LocalAlloc+0x75
0000000029ddf920 000007fefa36c187 TmUmEvt64!TmmonDestoryAddonObject+0x63d
0000000029ddf970 000007fefa36bbf1 TmUmEvt64!TmmonDestoryAddonObject+0x847
0000000029ddf9f0 0000000074862cd3 TmUmEvt64!TmmonDestoryAddonObject+0x2b1
0000000029ddfac0 00000000748629f4 tmmon64+0xe2cd3
0000000029ddfb80 00000000747b0aab tmmon64+0xe29f4
0000000029ddfbf0 0000000000467f5e tmmon64+0x30aab
0000000029ddfcc0 000000c0005c4000 compile+0x67f5e
0000000029ddfcc8 0000000000002000 0xc0005c4000
0000000029ddfcd0 0000000000001000 0x2000
0000000029ddfcd8 0000000000000004 0x1000
0000000029ddfce0 0000000000432b66 0x4
0000000029ddfce8 0000000076d914cf compile+0x32b66
0000000029ddfcf0 000000000000016c kernel32!GetTickCount+0x1f
0000000029ddfcf8 00000000ffffffff 0x16c
0000000029ddfd00 0000000000000000 0xffffffff

.  7  Id: 2c08.2f14 Suspend: 1 Teb: 000007fffffaa000 Unfrozen
Start: compile+0x68330 (0000000000468330)
Priority: 0  Priority class: 32  Affinity: f
000000002a01fb08 000007fefcd610ac ntdll!ZwWaitForSingleObject+0xa
000000002a01fb10 0000000000467f5e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002a01fbb0 000000c000080280 compile+0x67f5e
000000002a01fbb8 0000000001e7b5c0 0xc000080280
000000002a01fbc0 0000000000000000 0x1e7b5c0

.  8  Id: 2c08.2f64 Suspend: 1 Teb: 000007fffffa8000 Unfrozen
Start: ntdll!RtlDestroyHandleTable+0x270 (0000000076faf6f0)
Priority: 0  Priority class: 32  Affinity: f
000000002a68f3a8 0000000076fd8f58 ntdll!ZwWaitForSingleObject+0xa
000000002a68f3b0 0000000076fd8e54 ntdll!RtlDeNormalizeProcessParams+0x5a8
000000002a68f460 0000000076fde0b5 ntdll!RtlDeNormalizeProcessParams+0x4a4
000000002a68f490 0000000076fdddd8 ntdll!RtlAllocateHeap+0x455
000000002a68f670 0000000076fba762 ntdll!RtlAllocateHeap+0x178
000000002a68f780 000007fefcd62898 ntdll!RtlUserThreadStart+0x222
000000002a68f7c0 000007fefa38ae5e KERNELBASE!FlsSetValue+0x168
000000002a68f7f0 000007fefa38a152 TmUmEvt64!GetInterface+0x1e50e
000000002a68f820 000007fefa389c2d TmUmEvt64!GetInterface+0x1d802
000000002a68f850 000007fefa385c19 TmUmEvt64!GetInterface+0x1d2dd
000000002a68f880 000007fefa385fc9 TmUmEvt64!GetInterface+0x192c9
000000002a68f8b0 000007fefa3861f9 TmUmEvt64!GetInterface+0x19679
000000002a68f8e0 0000000076fba719 TmUmEvt64!GetInterface+0x198a9
000000002a68f940 0000000076fba46f ntdll!RtlUserThreadStart+0x1d9
000000002a68fa40 0000000076fba36e ntdll!LdrInitializeThunk+0x10f
000000002a68fab0 0000000000000000 ntdll!LdrInitializeThunk+0xe

.  9  Id: 2c08.2b84 Suspend: 1 Teb: 000007fffffa6000 Unfrozen
Start: ntdll!DbgUiRemoteBreakin (0000000077082dd0)
Priority: 0  Priority class: 32  Affinity: f
000000002a88ff28 0000000077082e08 ntdll!DbgBreakPoint
000000002a88ff30 0000000076d859cd ntdll!DbgUiRemoteBreakin+0x38
000000002a88ff60 0000000076fba561 kernel32!BaseThreadInitThunk+0xd
000000002a88ff90 0000000000000000 ntdll!RtlUserThreadStart+0x21



Alex

alexbrainman

comment created time in a month

issue commentgolang/go

can't sign a copyright-transfer agreement, sorry. That's the only reason I don't make CLs.

Fair enough. Then I won't look at any of your code postings.

Alex

comment created time in a month

issue commentgolang/go

Well you could FindNextFile() in a loop for i < seek_pos?

I don't understand what you are proposing. Please, send a CL with the change, if you know what to do.

Thank you.

Alex

comment created time in a month

issue commentgolang/go

Maybe it's worthwhile to consider exposing at least some types, vars, and consts from the packages under x/sys/*/?

Sure. What do you suggest?

And, do not forget, this is open source code, so you can copy whatever you like and put it anywhere you like.

I'm not referring to any functions that would actually require syscalls, obviously.

At lease you agree with me on this point.

Then I consider windows.GUID should only be useful to call actual syscalls. It should not be used for anything else. If you want to use windows.GUID for anything else, you need different type. It is fine, if you copy the code.

Alex

dcormier

comment created time in a month

issue commentgolang/go

Do you know what "tmmon64" and "TmUmEvt64" are?

This is not my computer, so I cannot properly poke at it. But I suspect this computer has some standard anrivirus software installed. And these kinds of software often would install their code to intercept real Win32 API calls.

Maybe we just need to hold the suspendLock for longer (though I don't have a theory for why this would be). What happens if you move the unlock(&suspendLock) in preemptM to the very bottom of the function?

I won't see her on the weekend. But I will try it Monday or Tuesday. Also please show diff of your suggested change, because I don't trust myself with reading your mind.

Thank you.

Alex

alexbrainman

comment created time in a month

issue commentgolang/go

Could File.Seek(0, os.SeekStart) be mapped to FindFirstFile() ?

Yes. But people will still complain that File.Seel(1, os.SeekStart) fails. So, lets not pretend.

NtQueryDirectoryFile

Yes, it might work. But is all extra new code worth ability to File.Seek?

Alex

comment created time in a month

issue commentgolang/go

Could you confirm that through some other process explorer and paste your report into a new issue if there's more than one thread in the hung process?

Done. See #36492.

Alex

bcmills

comment created time in a month

issue openedgolang/go

I have Windows 7 computer - windows/amd64.

I am building Go from source using go1.4 as bootstrap.

I am using commit 56d6b87

I am running make.bat command.

### What did you expect to see?

I expected make.bat command finish successfully.

### What did you see instead?

make.bat never finishes. It hangs, for example, like this:

c:\Users\alexb\dev\go\src>make
Building Go cmd/dist using c:\users\alexb\dev\\go1.4
Building Go toolchain1 using c:\users\alexb\dev\\go1.4.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.



I used process explorer https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer to examine process tree, and that is what I see:

I also used WinDbg to attach to go_bootstrap.exe (pid 9788) and print stacks of all its threads. And that is what I see:

0:019> !uniqstack

.  0  Id: 263c.23d8 Suspend: 1 Teb: 000007fffffde000 Unfrozen
Start: go_bootstrap+0x648e0 (00000000004648e0)
Priority: 0  Priority class: 32  Affinity: f
000000000022fae8 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000000022faf0 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000000022fb90 0000000000b37be0 go_bootstrap+0x6494e
000000000022fb98 00000000004270d3 go_bootstrap+0x737be0
000000000022fba0 0000000000000000 go_bootstrap+0x270d3

.  1  Id: 263c.314c Suspend: 1 Teb: 000007fffffdc000 Unfrozen
Priority: 0  Priority class: 32  Affinity: f
000000000302fcb8 000007fefcc71430 ntdll!ZwWaitForMultipleObjects+0xa
000000000302fcc0 0000000076df06c0 KERNELBASE!GetCurrentProcess+0x40
000000000302fdc0 00000000747aeed4 kernel32!WaitForMultipleObjects+0xb0
000000000302fe50 000000007478ad07 tmmon64+0xaeed4
000000000302ff00 000000007478aeae tmmon64+0x8ad07
000000000302ff30 0000000076df59cd tmmon64+0x8aeae
000000000302ff60 0000000076f2a561 kernel32!BaseThreadInitThunk+0xd
000000000302ff90 0000000000000000 ntdll!RtlUserThreadStart+0x21

.  2  Id: 263c.2d5c Suspend: 1 Teb: 000007fffffda000 Unfrozen
Start: ntdll!RtlDestroyHandleTable+0x270 (0000000076f1f6f0)
Priority: 0  Priority class: 32  Affinity: f
000000000322fcc8 0000000076f1ed15 ntdll!ZwWaitForWorkViaWorkerFactory+0xa
000000000322fcd0 0000000076df59cd ntdll!RtlValidateHeap+0x155
000000000322ff60 0000000076f2a561 kernel32!BaseThreadInitThunk+0xd
000000000322ff90 0000000000000000 ntdll!RtlUserThreadStart+0x21

.  3  Id: 263c.e74 Suspend: 1 Teb: 000007fffffd6000 Unfrozen
Start: ntdll!TpIsTimerSet+0x8b0 (0000000076f1a280)
Priority: 0  Priority class: 32  Affinity: f
000000000342fcb8 0000000076f1a3c7 ntdll!ZwWaitForMultipleObjects+0xa
000000000342fcc0 0000000076df59cd ntdll!TpIsTimerSet+0x9f7
000000000342ff60 0000000076f2a561 kernel32!BaseThreadInitThunk+0xd
000000000342ff90 0000000000000000 ntdll!RtlUserThreadStart+0x21

.  4  Id: 263c.196c Suspend: 1 Teb: 000007fffffd4000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
0000000028e5eb68 0000000076f48f58 ntdll!ZwWaitForSingleObject+0xa
0000000028e5eb70 0000000076f48e54 ntdll!RtlDeNormalizeProcessParams+0x5a8
0000000028e5ec20 000007fefa3b7f0b ntdll!RtlDeNormalizeProcessParams+0x4a4
0000000028e5ec50 000007fefa3b8504 TmUmEvt64+0x17f0b
0000000028e5eeb0 000007fefa3b8c96 TmUmEvt64+0x18504
0000000028e5ef10 000007fefa4565ca TmUmEvt64+0x18c96
0000000028e5efa0 000007fefa455f8e TmUmEvt64+0xb65ca
0000000028e5f000 000007fefa410686 TmUmEvt64+0xb5f8e
0000000028e5f150 000007fefa439730 TmUmEvt64+0x70686
0000000028e5f260 000000007472f146 TmUmEvt64+0x99730
0000000028e5f4a0 00000000747e2d7d tmmon64+0x2f146
0000000028e5f580 00000000747e29f4 tmmon64+0xe2d7d
0000000028e5f640 0000000074733748 tmmon64+0xe29f4
0000000028e5f6b0 000007fefcc77c3f tmmon64+0x33748
0000000028e5f780 000000000046494e KERNELBASE!ResumeThread+0xf
0000000028e5f7b0 ffffffffffffffff go_bootstrap+0x6494e
0000000028e5f7b8 0000000000000001 0xffffffffffffffff
0000000028e5f7c0 ffffffffffffffff 0x1
0000000028e5f7c8 0000000028e5f928 0xffffffffffffffff
0000000028e5f7d0 0000000000000000 0x28e5f928

.  5  Id: 263c.3204 Suspend: 1 Teb: 000007fffffae000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002905fb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000002905fb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002905fbb0 000000c00002c980 go_bootstrap+0x6494e
000000002905fbb8 0000000000000000 0xc00002c980

.  6  Id: 263c.32f0 Suspend: 1 Teb: 000007fffffac000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002925fb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000002925fb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002925fbb0 000000c00002cd00 go_bootstrap+0x6494e
000000002925fbb8 0000000000000164 0xc00002cd00
000000002925fbc0 7fffffff00000000 0x164
000000002925fbc8 0000000000000160 0x7fffffff00000000
000000002925fbd0 000000c0004c6300 0x160
000000002925fbd8 000000002925fcc0 0xc0004c6300
000000002925fbe0 000000000043daa2 0x2925fcc0
000000002925fbe8 0000000000b36bf8 go_bootstrap+0x3daa2
000000002925fbf0 0000000000000000 go_bootstrap+0x736bf8

.  7  Id: 263c.29d8 Suspend: 1 Teb: 000007fffffaa000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002945fb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000002945fb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002945fbb0 000000c000080280 go_bootstrap+0x6494e
000000002945fbb8 0000000000000000 0xc000080280

.  8  Id: 263c.104c Suspend: 1 Teb: 000007fffffa8000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
0000000029a8fb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
0000000029a8fb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
0000000029a8fbb0 000000c000206280 go_bootstrap+0x6494e
0000000029a8fbb8 0000000000000000 0xc000206280

.  9  Id: 263c.31b4 Suspend: 1 Teb: 000007fffffa6000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
0000000029ccfb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
0000000029ccfb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
0000000029ccfbb0 000000c000207080 go_bootstrap+0x6494e
0000000029ccfbb8 0000000000000000 0xc000207080

. 10  Id: 263c.2b14 Suspend: 1 Teb: 000007fffffa4000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
0000000029eceb68 0000000076f48f58 ntdll!ZwWaitForSingleObject+0xa
0000000029eceb70 0000000076f48e54 ntdll!RtlDeNormalizeProcessParams+0x5a8
0000000029ecec20 000007fefa3b7f0b ntdll!RtlDeNormalizeProcessParams+0x4a4
0000000029ecec50 000007fefa3b8504 TmUmEvt64+0x17f0b
0000000029eceeb0 000007fefa3b8c96 TmUmEvt64+0x18504
0000000029ecef10 000007fefa4565ca TmUmEvt64+0x18c96
0000000029ecefa0 000007fefa455f8e TmUmEvt64+0xb65ca
0000000029ecf000 000007fefa410686 TmUmEvt64+0xb5f8e
0000000029ecf150 000007fefa439730 TmUmEvt64+0x70686
0000000029ecf260 000000007472f146 TmUmEvt64+0x99730
0000000029ecf4a0 00000000747e2d7d tmmon64+0x2f146
0000000029ecf580 00000000747e29f4 tmmon64+0xe2d7d
0000000029ecf640 0000000074733748 tmmon64+0xe29f4
0000000029ecf6b0 000007fefcc77c3f tmmon64+0x33748
0000000029ecf780 000000000046494e KERNELBASE!ResumeThread+0xf
0000000029ecf7b0 ffffffffffffffff go_bootstrap+0x6494e
0000000029ecf7b8 0000000000000000 0xffffffffffffffff

. 11  Id: 263c.30b4 Suspend: 1 Teb: 000007fffffa2000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002a10fb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000002a10fb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002a10fbb0 000000c000195400 go_bootstrap+0x6494e
000000002a10fbb8 0000000000000009 0xc000195400
000000002a10fbc0 0000000000000000 0x9

. 12  Id: 263c.30fc Suspend: 1 Teb: 000007fffffa0000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002a30f248 0000000076f48f58 ntdll!ZwWaitForSingleObject+0xa
000000002a30f250 0000000076f48e54 ntdll!RtlDeNormalizeProcessParams+0x5a8
000000002a30f300 000007fefa3b7f0b ntdll!RtlDeNormalizeProcessParams+0x4a4
000000002a30f330 000007fefa3b8504 TmUmEvt64+0x17f0b
000000002a30f590 000007fefa3b8c96 TmUmEvt64+0x18504
000000002a30f5f0 000007fefa4565ca TmUmEvt64+0x18c96
000000002a30f680 000007fefa455f8e TmUmEvt64+0xb65ca
000000002a30f6e0 000007fefa410686 TmUmEvt64+0xb5f8e
000000002a30f830 000007fefa4363b0 TmUmEvt64+0x70686
000000002a30f940 000000007472f146 TmUmEvt64+0x963b0
000000002a30fb80 00000000747e2d7d tmmon64+0x2f146
000000002a30fc60 00000000747e29f4 tmmon64+0xe2d7d
000000002a30fd20 0000000074732869 tmmon64+0xe29f4
000000002a30fd90 000000000046494e tmmon64+0x32869
000000002a30fe60 000000c000306100 go_bootstrap+0x6494e
000000002a30fe68 000000c00016b840 0xc000306100
000000002a30fe70 000000002a30fe60 0xc00016b840
000000002a30fe78 0000000000000000 0x2a30fe60

. 13  Id: 263c.13a0 Suspend: 2 Teb: 000007fffff9e000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002a50ec50 000007fefa3b8504 TmUmEvt64+0x17fa0
000000002a50eeb0 000007fefa3b8c96 TmUmEvt64+0x18504
000000002a50ef10 000007fefa4565ca TmUmEvt64+0x18c96
000000002a50efa0 000007fefa455f8e TmUmEvt64+0xb65ca
000000002a50f000 000007fefa410686 TmUmEvt64+0xb5f8e
000000002a50f150 000007fefa439730 TmUmEvt64+0x70686
000000002a50f260 000000007472f146 TmUmEvt64+0x99730
000000002a50f4a0 00000000747e2d7d tmmon64+0x2f146
000000002a50f580 00000000747e29f4 tmmon64+0xe2d7d
000000002a50f640 0000000074733748 tmmon64+0xe29f4
000000002a50f6b0 000007fefcc77c3f tmmon64+0x33748
000000002a50f780 000000000046494e KERNELBASE!ResumeThread+0xf
000000002a50f7b0 ffffffffffffffff go_bootstrap+0x6494e
000000002a50f7b8 0000000000000000 0xffffffffffffffff

. 15  Id: 263c.1c34 Suspend: 1 Teb: 000007fffff9a000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002a90faf8 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000002a90fb00 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002a90fba0 000000c0003b9400 go_bootstrap+0x6494e
000000002a90fba8 fffffffffff85ee0 0xc0003b9400
000000002a90fbb0 0000000000000000 0xfffffffffff85ee0

. 16  Id: 263c.2530 Suspend: 1 Teb: 000007fffff98000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002ab0fb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000002ab0fb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002ab0fbb0 000000c0003b9780 go_bootstrap+0x6494e
000000002ab0fbb8 0000000000000009 0xc0003b9780
000000002ab0fbc0 0000000000000000 0x9

. 17  Id: 263c.3114 Suspend: 1 Teb: 000007fffff96000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002ad0fb08 000007fefcc710ac ntdll!ZwWaitForSingleObject+0xa
000000002ad0fb10 000000000046494e KERNELBASE!WaitForSingleObjectEx+0x9c
000000002ad0fbb0 000000c000508d00 go_bootstrap+0x6494e
000000002ad0fbb8 0000000000000000 0xc000508d00

. 18  Id: 263c.3344 Suspend: 3 Teb: 000007fffff94000 Unfrozen
Start: go_bootstrap+0x64d00 (0000000000464d00)
Priority: 0  Priority class: 32  Affinity: f
000000002af0fb08 000007fefcc71430 ntdll!ZwWaitForMultipleObjects+0xa
000000002af0fb10 0000000076df06c0 KERNELBASE!GetCurrentProcess+0x40
000000002af0fc10 000000000046494e kernel32!WaitForMultipleObjects+0xb0
000000002af0fca0 000007fffff94000 go_bootstrap+0x6494e
000000002af0fca8 0000000000000000 0x7fffff94000

. 19  Id: 263c.3128 Suspend: 1 Teb: 000007fffff92000 Unfrozen
Start: ntdll!DbgUiRemoteBreakin (0000000076ff2dd0)
Priority: 0  Priority class: 32  Affinity: f
000000002b10ff28 0000000076ff2e08 ntdll!DbgBreakPoint
000000002b10ff30 0000000076df59cd ntdll!DbgUiRemoteBreakin+0x38
000000002b10ff60 0000000076f2a561 kernel32!BaseThreadInitThunk+0xd
000000002b10ff90 0000000000000000 ntdll!RtlUserThreadStart+0x21

Duplicate callstacks: 1 (windbg thread #s follow):
14


I was able to use Delve to examine this bug once (see https://github.com/golang/go/issues/35775#issuecomment-571967772), but not anymore. Delve just fails to attach now.

I can reproduce this pretty reliably on this particular computer - make.bat never completes. Sometimes it hangs in go_bootstrap.exe and sometimes in compile.exe. Sometimes there are more than single hung compile.exe.

I can make problem go away, if I change source code to have runtime.preemptMSupported set to false.

/cc @aclements

Alex

created time in a month

issue commentgolang/go

I'm not sure we want to do this.

I agree. x/sys/windows is started as Windows version of syscall package. And syscall.GUID makes no sense anywhere but Windows.

Alex

dcormier

comment created time in a month

issue commentgolang/go

I also have make.bat reliably hanging on a Windows 7 computer here. I run make.bat (I use GOARCH=amd64) on top of

commit f376b8510ed7884c69a09fbcf61418f7285f2787 (origin/master, origin/HEAD, mas
Author: Emmanuel T Odeke <emmanuel@orijtech.com>
Date:   Thu Jan 2 02:00:55 2020 -0800

all: fix invalid invocations of Fatalf in goroutines

Found by running the go vet pass 'testinggoroutine' that
I started in CL 212920.

Change-Id: Ic9462fac85dbafc437fe4a323b886755a67a1efa
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>


and this is how far it gets

As you can see, there is stuck go_bootstrap.exe there. I used Delve to see what the process does, and this is what I see:

(dlv) threads
md64.s:805 crypto/sha256.block
(dlv) goroutines -t
Goroutine 1 - User: c:/Users/alexb/dev/go/src/runtime/sema.go:56 sync.runtime_
Semacquire (0x447179)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x000000000044754e in runtime.goparkunlock
at c:/Users/alexb/dev/go/src/runtime/proc.go:310
2  0x000000000044754e in runtime.semacquire1
at c:/Users/alexb/dev/go/src/runtime/sema.go:144
3  0x0000000000447179 in sync.runtime_Semacquire
at c:/Users/alexb/dev/go/src/runtime/sema.go:56
4  0x000000000047e4bb in sync.(*WaitGroup).Wait
at c:/Users/alexb/dev/go/src/sync/waitgroup.go:130
5  0x00000000006a6735 in cmd/go/internal/work.(*Builder).Do
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:187
6  0x00000000006a08c6 in cmd/go/internal/work.InstallPackages
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/build.go:605
7  0x000000000069fc3d in cmd/go/internal/work.runInstall
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/build.go:516
8  0x000000000077d0d6 in main.main
at c:/Users/alexb/dev/go/src/cmd/go/main.go:189
9  0x0000000000438212 in runtime.main
at c:/Users/alexb/dev/go/src/runtime/proc.go:203
10  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 2 - User: c:/Users/alexb/dev/go/src/runtime/proc.go:305 runtime.gopa
rk (0x4385e5)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x0000000000438485 in runtime.goparkunlock
at c:/Users/alexb/dev/go/src/runtime/proc.go:310
2  0x0000000000438485 in runtime.forcegchelper
at c:/Users/alexb/dev/go/src/runtime/proc.go:253
3  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 3 - User: c:/Users/alexb/dev/go/src/runtime/proc.go:305 runtime.gopa
rk (0x4385e5)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x00000000004232bf in runtime.goparkunlock
at c:/Users/alexb/dev/go/src/runtime/proc.go:310
2  0x00000000004232bf in runtime.bgsweep
at c:/Users/alexb/dev/go/src/runtime/mgcsweep.go:89
3  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 4 - User: c:/Users/alexb/dev/go/src/runtime/proc.go:305 runtime.gopa
rk (0x4385e5)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x0000000000421718 in runtime.goparkunlock
at c:/Users/alexb/dev/go/src/runtime/proc.go:310
2  0x0000000000421718 in runtime.scavengeSleep
at c:/Users/alexb/dev/go/src/runtime/mgcscavenge.go:203
3  0x0000000000421a32 in runtime.bgscavenge
at c:/Users/alexb/dev/go/src/runtime/mgcscavenge.go:302
4  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 5 - User: c:/Users/alexb/dev/go/src/runtime/proc.go:305 runtime.gopa
rk (0x4385e5)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x0000000000418b91 in runtime.goparkunlock
at c:/Users/alexb/dev/go/src/runtime/proc.go:310
2  0x0000000000418b91 in runtime.runfinq
at c:/Users/alexb/dev/go/src/runtime/mfinal.go:175
3  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 39 - User: :0 ??? (0x77ace306) (thread 13956)
0  0x0000000077ace306 in ???
at ?:-1
error: Only part of a ReadProcessMemory or WriteProcessMemory reques
t was completed.
(truncated)
Goroutine 54 - User: c:/Users/alexb/dev/go/src/runtime/proc.go:305 runtime.gop
ark (0x4385e5)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x000000000041c6f4 in runtime.gcBgMarkWorker
at c:/Users/alexb/dev/go/src/runtime/mgc.go:1880
2  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 55 - User: :0 ??? (0x77acbd7a) (thread 3912)
0  0x0000000077acbd7a in ???
at ?:-1
error: Only part of a ReadProcessMemory or WriteProcessMemory reques
t was completed.
(truncated)
Goroutine 56 - User: :0 ??? (0x77acbd7a) (thread 7680)
0  0x0000000077acbd7a in ???
at ?:-1
error: Only part of a ReadProcessMemory or WriteProcessMemory reques
t was completed.
(truncated)
Goroutine 991 - User: c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:1
68 cmd/go/internal/work.(*Builder).Do.func3 (0x6dc154)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x00000000004469e8 in runtime.selectgo
at c:/Users/alexb/dev/go/src/runtime/select.go:316
2  0x00000000006dc154 in cmd/go/internal/work.(*Builder).Do.func3
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:168
3  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 992 - User: c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:1
68 cmd/go/internal/work.(*Builder).Do.func3 (0x6dc154)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x00000000004469e8 in runtime.selectgo
at c:/Users/alexb/dev/go/src/runtime/select.go:316
2  0x00000000006dc154 in cmd/go/internal/work.(*Builder).Do.func3
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:168
3  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 993 - User: c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:1
68 cmd/go/internal/work.(*Builder).Do.func3 (0x6dc154)
0  0x00000000004385e5 in runtime.gopark
at c:/Users/alexb/dev/go/src/runtime/proc.go:305
1  0x00000000004469e8 in runtime.selectgo
at c:/Users/alexb/dev/go/src/runtime/select.go:316
2  0x00000000006dc154 in cmd/go/internal/work.(*Builder).Do.func3
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:168
3  0x0000000000462c91 in runtime.goexit
at c:/Users/alexb/dev/go/src/runtime/asm_amd64.s:1375
Goroutine 1042 - User: c:/Users/alexb/dev/go/src/crypto/sha256/sha256block_amd
0  0x00000000005b6351 in crypto/sha256.block
at c:/Users/alexb/dev/go/src/crypto/sha256/sha256block_amd64.s:805
1  0x00000000005b25f2 in crypto/sha256.(*digest).Write
at c:/Users/alexb/dev/go/src/crypto/sha256/sha256.go:198
2  0x00000000004a3f2e in io.(*multiWriter).Write
at c:/Users/alexb/dev/go/src/io/multi.go:60
3  0x00000000004a38c2 in io.copyBuffer
at c:/Users/alexb/dev/go/src/io/io.go:407
4  0x00000000004a3621 in io.Copy
at c:/Users/alexb/dev/go/src/io/io.go:364
5  0x00000000004a3621 in io.CopyN
at c:/Users/alexb/dev/go/src/io/io.go:340
6  0x00000000005be33a in cmd/go/internal/cache.(*Cache).copyFile
at c:/Users/alexb/dev/go/src/cmd/go/internal/cache/cache.go:490
7  0x00000000005bdd43 in cmd/go/internal/cache.(*Cache).put
at c:/Users/alexb/dev/go/src/cmd/go/internal/cache/cache.go:428
8  0x00000000006a5897 in cmd/go/internal/cache.(*Cache).Put
at c:/Users/alexb/dev/go/src/cmd/go/internal/cache/cache.go:403
9  0x00000000006a5897 in cmd/go/internal/work.(*Builder).updateBuildID
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/buildid.go:706
10  0x00000000006aaf34 in cmd/go/internal/work.(*Builder).build
at c:/Users/alexb/dev/go/src/cmd/go/internal/work/exec.go:783
(truncated)
[13 goroutines]
(dlv)


I also tried reproducing this on another Windows 7 (386 this time) running in VM, and I could not reproduce it. I am not sure what is special about this computer that it fails. But, I guess, if it is a race, then anything is possible. Anything can push it over the age.

Also I can make.bat pass, if I set runtime.preemptMSupported to false.

Let me know, if you want me to debug something. I can reproduce it pretty much every time.

Sorry all this details are vague. That is only free time I had.

Alex

bcmills

comment created time in a month

issue commentgolang/go

@bsamek I can reproduce it here, thank you very much.

The problem is that

os.RemoveAll("foo")


calls

os.Remove(foo\bar\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb)


which calls

syscall.RemoveDirectory(foo\bar\aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa\bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb)


which fails, because syscalls don't allow for relative path to be longer than 256 chars. But syscall.RemoveDirectory should succeeds, because directory exists and can be deleted.

Path is converted with os.fixLongPath in os.Remove, but os.fixLongPath does not handle relative path, and returns them as is.

Alex

bsamek

comment created time in 2 months

issue commentgolang/go

I could reproduce this problem, but only in very special circumstances.

I could not reproduce the problem, if I do file time stamp reading from Windows. I used both os.File.Stat and shelling out with cmd.exe /c dir ... for every test program I tried, and time stamps are as expected. I could only reproduce this problem, if I create file from Windows Go program, and then use stat or ls -l command on Linux to view created file times.

I managed reproduce the problem with this program https://play.golang.org/p/OVLuYilFE7E

But, if I add f.Close() after line 39, the problem goes away.

I can also make problem go away, if I add

        err = f.Sync()
if err != nil {
t.Fatal(err)
}


after line 39.

I don't know @spf13 what your problem is (you did not provide your program), but I suspect that Samba updates modtime when file is written to, but writes might be buffered on the client side. On the other hand, Samba Windows SetFileTime API implementation (Go uses SetFileTime to implement os.Chtimes) is not buffered.

I am not sure if there is anything more we could do here.

Alex

spf13

comment created time in 2 months

issue commentgolang/go

I would call NUL a MS-DOS device name. It is only supported in userland DLLs for compatiblity reason.

If you meant Win32 API, then, yes, that is Go on Windows support.

Alex

zhangyoufu

comment created time in 2 months

issue commentgolang/go

gcc invoked as, and create this NUL file via NtCreateFile. Caller is https://github.com/msys2/Cygwin/blob/bede85ba6d90b9383f3c83a6e99152284ca90f6a/winsup/cygwin/path.cc#L3011.

@zhangyoufu Go doesn't support Cygwin.

Cygwin tools use paths that Windows don't support. Windows Go port only works with normal Windows paths.

Alex

zhangyoufu

comment created time in 2 months

issue commentgolang/go

Do you want to send change you described? This https://golang.org/doc/contribute.html is how to contribute.

Thank you.

Alex

GreenLightning

comment created time in 2 months

issue commentgolang/go

I'm expecting a local device path like \\.\NUL should work for more situation than NUL.

Sorry, but I am not convinced that using NUL as gcc output file is a problem. I am also not convinced that \\.\NUL is always better than NUL. I prefer to leave everything as is.

Alex

zhangyoufu

comment created time in 2 months

issue commentgolang/go

Go use FindFirstFile, FindNextFile and FindClose Windows API to implement directory reading. The API does not provide functionality to seek to random position of the directory. I don't know what the alternatives are, if any.

Alex

comment created time in 2 months

issue commentgolang/go

@ncw I can reproduce your problem.

I briefly looked at your code, and I cannot see why returned result is limited to 254 chars.

Maybe @mattn know.

Alex

ncw

comment created time in 2 months

issue commentgolang/go

@zhangyoufu I use gcc as suggested at

https://github.com/golang/go/issues/35006#issuecomment-545363716

and everything works for me:

C:\>gcc --version
gcc (GCC) 9.2.1 20190831
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

C:\>gcc -fno-caret-diagnostics -c -x c -o NIL
gcc: warning: '-x c' after last input file has no effect
gcc: error: unrecognized command line option '-fno-caret-diagnostics'
gcc: fatal error: no input files
compilation terminated.

C:\>gcc -fno-caret-diagnostics -c -x c -o nil
gcc: warning: '-x c' after last input file has no effect
gcc: error: unrecognized command line option '-fno-caret-diagnostics'
gcc: fatal error: no input files
compilation terminated.

C:\>


I suggest you change your gcc. Would that work for you?

Alex

zhangyoufu

comment created time in 2 months

issue commentgolang/go

Looking at testRuntimeTypeAttr source, I notice that you open file

f := gobuild(t, dir, prog, flags)

and then execute the file

... exec.Command(f.path).CombinedOutput()

I am surprised that Windows allows you to execute opened file.

Perhaps unrelated, but I also noticed that file f is never closed. Why?

Alex

bcmills

comment created time in 2 months

issue commentgolang/go

All windows syscall changes LGTM.

Alex

toothrot

comment created time in 2 months

issue commentgolang/go

Here's another idea. We could decide to try #32088 by landing the change on day 1 of the Go 1.15 cycle and then wait the 3 months to see if any problems are reported before deciding to keep it. ...

If we did that trial period, would that ease some of your concerns, @alexbrainman and @mattn?

Not really. Nothing you said in https://github.com/golang/go/issues/32088#issuecomment-537590826 changed.

Alex

rsc

comment created time in 2 months

issue commentgolang/go

Although I don't understand the reference to CL 198417.

Issue #31528 was fixed by CL 191957. But after CL 191957 was submitted, Austin suggested an improvement to it. And Jason implemented the improvement in CL 198417.

I could have selected CL 191957 to test the issue. Any commit after CL 191957 is good. I chose CL 198417.

Just to confirm: did you run those tests on current tip?

I did not test current tip. I think issue #31528 is still fixed on current tip. But issue is broken again on CL 210437.

I see three options here:

@aclements I don't have time to invest into this. Whatever you decide, I will be happy.

What you see in that screencast is that Docker uses "program time", whereas real Windows uses "real time".

@zx2c4 thank you for confirming that Docker uses "program time".

THIS MEANS THAT THE ORIGINAL SIMPLE COMMIT FIXES THE DOCKER ISSUE

Yes. CL 208317 will allow Docker to run. But Docker time behavior is still different from real Windows. And we need to bring them in line somehow.

So at this point, I'd strongly recommend merging that commit, closing this issue, and starting a new discussion on "real time" vs "program time" and new APIs for Go 1.15.

Sounds reasonable to me.

Alex

networkimprov

comment created time in 2 months

issue commentgolang/go

I hear Microsoft is EOL'ing Windows 7, so maybe we don't have to worry about this much longer. :)

That is what Microsoft says. But about half of my customers still use Windows 7 - their computers do the job, so why changing. I also use Windows 7 laptop at work - that is what I have been given few years ago, and I have no reason changing it.

That would be great.

I build this https://play.golang.org/p/xNHWgBD9k5M program (@zx2c4 wrote that program) on top of CL 198417. And then I run it on WIndows 10 laptop. I run the program, waited until Sleeping ... message come up, pressed enter key, closed laptop lid, waited for about 5 seconds, opened laptop and waited for program to print resulting message and exit. I run this program about 3 times and every time I got this output

Sleeping process for 1m0s. Suspend your computer into S3 immediately and then wake it up 4 seconds later.
Done sleeping.
All went well.


Then I run same program built on top of CL 210437, 3 times again. And every time it failed with:

Sleeping process for 1m0s. Suspend your computer into S3 immediately and then wake it up 4 seconds later.
Done sleeping.
FAILURE. Diff: 3.5861624s


(the number varied at different tests, but they all around 3 seconds).

So I don't see that Ian's latest timer changes fix #31528. I did not have chance to do the same test on Windows 7. But Windows 7 was not broken (as per #31528), so I don't see what could change there, so I won't bother testing it.

So now I have a (cheap) Windows laptop and have Go set up on it so I can experiment and verify things.

Congratulation! Now you could debug things properly. I will let you try and play with GetQueuedCompletionStatus and GetQueuedCompletionStatusEx too. Feel free to ask for whatever help I can give you. I suspect you will have Windows 10, so if you want me to try something on Windows 7, I can do that.

Alex

networkimprov

comment created time in 2 months

issue commentgolang/go

Seems to me like it's just Alex and me doing the the majority of the heavy lifting as of late.

I don't think this is true. Austin just recently implemented preemption on Windows. He even had to do psychic debugging - debug new preemption code without Windows machine. And Ian always reviews Windows issues and code changes.

I think we're definitely interested in fixing the bug, but at the moment it seems to me that the best path forward is to revert back to the buggy but compatible state and then try to roll forward without breaking any existing projects.

What about https://go-review.googlesource.com/c/go/+/208317 ? If we can make this change to only work inside Docker, I think it is fine temporary measure until we have proper solution for Docker.

My understanding is QueryUnbiasedInterruptTime would change time.Sub to use "program time", which would make Windows 8 use program time for both duration and sleep.

You are correct, QueryUnbiasedInterruptTime does not account for laptop sleep time. From

https://docs.microsoft.com/en-us/windows/win32/api/realtimeapiset/nf-realtimeapiset-queryunbiasedinterrupttime

The unbiased interrupt-time count does not include time the system spends in sleep or hibernation.

And there is a version that accounts for laptop sleep time - QueryInterruptTime.

And there are even precise versions of these - QueryUnbiasedInterruptTimePrecise and QueryInterruptTimePrecise. But precise versions are about 5 times slower than non precise versions ( see https://github.com/golang/go/issues/31528#issuecomment-500270601 for details). Also precise versions are only available on Windows 10.

But what happens to time.Sleep on Windows < 8? Wouldn't that still be "real time", causing a different kind of skew on older Windows machines?

If we use WaitForSingleObject to wait, WaitForSingleObject uses "real time" on Windows 7, and "program time" on Windows 10.

I note that as far as I can tell the patch in CL 191957 has no effect on the current timer code. It only affects goroutines sleeping in semasleep, which no longer happens for timers.

This is a news to me. I suppose I should test #31528 with https://golang.org/cl/210437 applied and see what happens.

Does anybody know how GetQueuedCompletionStatus and GetQueuedCompletionStatusEx handle their timeouts when the system goes into low power mode?

I do not know. Let me know, if you want me to find out. I need to find appropriate laptop.

For 1.15 I think we should aim to have Windows timers behave like timers on non-Windows systems, which is to say that they should ignore suspend time.

What about #24595 ? What happens, if you decide to account for suspend time on Linux?

As there is a need for timers that do not ignore suspend time, @aclements has suggested that we should introduce time.SleepUntil and time.NewTimerAt (and perhaps time.At and time.AtFunc). These new functions will take a time.Time rather than time.Duration, and will wake up or send a value on a channel or call a function at the specified time. We should also see if we can fix time.Sub to report a duration that includes suspend time when taking the difference between a call to time.Now before the suspension and a call to time.Now after the system wakes up again.

Sounds good to me. But I am not big time user. @zx2c4 is.

Alex

networkimprov

comment created time in 2 months

issue commentgolang/go

we should be using on Windows is actually RtlGenRandom

The RtlGenRandom doco

https://docs.microsoft.com/en-us/windows/win32/api/ntsecapi/nf-ntsecapi-rtlgenrandom

suggests that we should use CryptGenRandom

It may be altered or unavailable in subsequent versions. Instead, use the CryptGenRandom function

@zx2c4 why do you prefer RtlGenRandom to current CryptGenRandom? I am fine with either.

Also, if we follow CryptGenRandom doco

https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom

This API is deprecated. New and existing software should start using Cryptography Next Generation APIs.

We should, probably, use BCryptGenRandom. No?

Alex

randomvariable

comment created time in 2 months

issue commentgolang/go

This will return us to the state in the left column above where on Windows >= 8, time.Sub is real time and time.Sleep is program time.

This is not good that time.Sub and time.Sleep will use different clocks. This is unexpected for Go users - see #31528 for an example.

@aclements you didn't address the suggestion to use QueryUnbiasedInterruptTime() to

Given Austin's decision to delay decision, I don't think we should be changing runtime.nanotime implementation until after Go Team decides what they want.

Alex

networkimprov

comment created time in 2 months

issue commentgolang/go

@networkimprov you seems to ignore Ian's argument.

The question is not what will happen with programs aware of the behavior change; it's what will happen with programs that are not aware. And programs that are not aware will not use the experimental opt-in.

We all spent a lot of time investigating this change. But people that will be affected by this change have very little knowledge about this matter. They won't bother trying your experiment.

The experimental period could be 6, 12, or 18mos.

I don't believe anything will change in 18 month. FILE_SHARE_DELETE will still be here affecting existing programs. If Microsoft were really interested in changing this behavior, they could have started ignoring FILE_SHARE_DELETE in latest versions of Windows. I don't see that is happening. But then we don't need to change anything in Go.

Also having experiment in Go release is too high price to pay for this feature. We already have github.com/alexbrainman/goissue34681 - perfectly fine solution to your problem.

Alex

rsc

comment created time in 2 months

issue commentgolang/go

Thank you for creating this issue.

Perhaps this is just a flake but failing to find getsockname in the Windows Socket DLL looks serious.

I agree, it looks like we could not find getsockname function in ws2_32.dll. The error is ERROR_PROC_NOT_FOUND.

The error does look unusual. I am not sure we can do anything about that. Maybe @zx2c4 has some bright ideas.

Alex

odeke-em

comment created time in 2 months

issue commentgolang/go

If the handler send to changesFromHandler channel after the break loop due to r.errno != 0, the handler goroutine and the unbuffered channel will be leaked.

I understand that sysErrNewThreadInCallback can hardly happen, and it is generally safe to assume that caller will exit after svc.Run returned. This is just a minor issue.

If r.errno != 0, then service is exiting, and it doesn't matter what happens to leaked handler goroutine or unbuffered channel.

1. svc.Run called os.LockOSThread(), but does not have defer os.UnlockOSThread()

Same. When StartServiceCtrlDispatcher returns, it means that service is about to exit. So it does not matter what happens to goroutine that calls os.LockOSThread().

2. service.close() is not referenced anywhere, and it should be removed because CloseHandle while WaitForSingleObject is an undefined behavior

Yes, service.close() is not called by anything. I don't remember why it is there. Perhaps it was used for debugging during development. I would rather leave it as is. It is just a couple lines of code. If you not insist.

Thank you.

Alex

zhangyoufu

comment created time in 2 months

issue commentgolang/go

I wonder if it it not an issue with compilation.

I don't know what the problem is.

I am using the following to compile the binary on Linux and then I'm executing it on Windows:

Same.

Alex

KyhleOhlinger

comment created time in 2 months

issue commentgolang/go

Possibly the runtime would not exit(2) when building as a DLL if that is possible and viable.

I agree, we should do that. This issue is similar to recent

where Go DLL would crash whole process when handling exception.

@martinzak-zaber do you want to send appropriate change? This

https://golang.org/doc/contribute.html

is how to contribute.

Alex

martinzak-zaber

comment created time in 2 months

issue commentgolang/go

@rillig I don't have Cygwin. I tried guessing your repro, but failed

c:\Users\Alex\dev\go\src>cd %GOPATH%

c:\Users\Alex\dev>cd src

c:\Users\Alex\dev\src>go get github.com/rillig/pkglint

c:\Users\Alex\dev\src>


Alex

rillig

comment created time in 3 months

issue commentgolang/go

@KyhleOhlinger I cannot reproduce your problem here.

I run https://play.golang.org/p/cco7-UO_j-E here, and I see this output

C:\>u:\test -test.v
=== RUN   TestALEX
TestALEX: registry_test.go:21: got="C:\\ProgramData\\Microsoft\\Windows Defender\\platform\\4.18.1909.6-0" gottype=1
--- PASS: TestALEX (0.01s)
PASS

C:\>

I also use Windows 10.

Alex
KyhleOhlinger

comment created time in 3 months

issue commentgolang/go

@alexbrainman and @mattn, we have someone who works on the kernel team at Microsoft saying that we should probably just set the flag unconditionally. Given this new information do you still think we should decline #32088?

If you are referring to @jstarks , I think he said the opposite - from https://github.com/golang/go/issues/34681#issuecomment-556218563

I agree with you and others that #32088 is the wrong choice for Go at this point. This is a change from my original position. ...

Alex

rsc

comment created time in 3 months

issue commentgolang/go

I'm not entirely sure where to put it though, let me know if you have thoughts.

I think before updating any documentation, you should just describe how to build Cgo code with MSVC here. For @mxplusb and @kesmit13 so they can try your changes.

For example, to use gcc to build Cgo code, I

1. adjust my %PATH% to include gcc.exe
2. build Go first by running make.bat from %GOROOT%\src directory
3. use standard go build and go install commands

How do I do the same with MSVC?

What I am not sure about is how we will filter the tests to be ran down to things that only MSVC could impact if we aren't running all tests twice. Potentially just limiting it to CGO related tests, but I believe there are CGO tests all over the place and I don't know if dist has a mechanism to filter those out?

I would start with %GOROOT%\misc\cgo directory. Whatever source you can build there you should build. Anything above that is up to you.

Alex

ghost

comment created time in 3 months

issue commentgolang/go

@rsc and @ianlancetaylor

The MINGW gcc our builders use is very very old. We need to decide on the replacement - see my comment https://github.com/golang/go/issues/35006#issuecomment-546573751 and whole thread in general.

What do you think?

Thank you.

Alex

alexbrainman

comment created time in 3 months

issue commentgolang/go

@mdempsky one last crash remaining after I apply

--- FAIL: TestPacketConn (0.00s)
panic: runtime error: unsafe pointer conversion [recovered]
panic: runtime error: unsafe pointer conversion

goroutine 203 [running]:
testing.tRunner.func1(0xc0003aab40)
c:/users/alex/dev/go/src/testing/testing.go:916 +0x60b
panic(0x6781a0, 0xc00039dea0)
c:/users/alex/dev/go/src/runtime/panic.go:961 +0x16b
syscall.WSASendto(0x278, 0xc0003c57b8, 0x100000000000001, 0xc0003c57a0, 0x0, 0x6ebec0, 0xc000371e80, 0xc0003c5770, 0x0, 0x0, ...)
c:/users/alex/dev/go/src/syscall/syscall_windows.go:876 +0x9b
internal/poll.(*FD).WriteTo.func2(0xc0003c5770, 0x77, 0xc00041fc00)
c:/users/alex/dev/go/src/internal/poll/fd_windows.go:869 +0x86
internal/poll.(*ioSrv).ExecIO(0x87fb68, 0xc0003c5770, 0x6a9b98, 0x26b54ef8, 0x4, 0x100000000000000)
c:/users/alex/dev/go/src/internal/poll/fd_windows.go:206 +0xbb
internal/poll.(*FD).WriteTo(0xc0003c5680, 0xc0003b1360, 0xf, 0xf, 0x6ebec0, 0xc000371e80, 0x0, 0x0, 0x0)
c:/users/alex/dev/go/src/internal/poll/fd_windows.go:868 +0x285
net.(*netFD).writeTo(0xc0003c5680, 0xc0003b1360, 0xf, 0xf, 0x6ebec0, 0xc000371e80, 0x10000001dcd6500, 0x0, 0x80)
c:/users/alex/dev/go/src/net/fd_windows.go:187 +0x76
net.(*UDPConn).writeTo(0xc0003a6fa8, 0xc0003b1360, 0xf, 0xf, 0xc0003c33b0, 0xc0003c5900, 0xbf6e961d56f1ac28, 0x11de793c1)
c:/users/alex/dev/go/src/net/udpsock_posix.go:80 +0xd8
net.(*UDPConn).WriteTo(0xc0003a6fa8, 0xc0003b1360, 0xf, 0xf, 0x6ec6c0, 0xc0003c33b0, 0x882200, 0xb, 0x698dac)
c:/users/alex/dev/go/src/net/udpsock.go:167 +0x9a
net.TestPacketConn(0xc0003aab40)
c:/users/alex/dev/go/src/net/packetconn_test.go:78 +0x62e
testing.tRunner(0xc0003aab40, 0x6aa408)
c:/users/alex/dev/go/src/testing/testing.go:954 +0xe3
created by testing.(*T).Run
c:/users/alex/dev/go/src/testing/testing.go:1005 +0x35e
FAIL    net     4.355s
FAIL


I am not sure, but I suspect the problem is conversion of *SockaddrInet4 to unsafe.Pointer to *RawSockaddrAny.

https://github.com/golang/go/blob/6f7b96f6cb5f5c156cf4e69b1fa1ec0e57b01677/src/syscall/syscall_windows.go#L859

Linux does something similar but the final pointer is *byte

https://github.com/golang/go/blob/6f7b96f6cb5f5c156cf4e69b1fa1ec0e57b01677/src/syscall/syscall_linux.go#L666

, so pointer checker does not complain.

How do I change code to avoid the crash?

Thank you.

Alex

mdempsky

comment created time in 3 months

issue commentgolang/go

@jstarks can you help with this issue? It would be sad, if windows-arm code gets removed just because we don't have the builder.

Thank you.

Alex

alexbrainman

comment created time in 3 months

issue commentgolang/go

I can find some time to make the change that Alex originally prototyped if no one else is available. But in the meantime, a patch release of Go has regressed programs running in containers, so should we consider reverting this and then apply a more appropriate change later?

I have no opinion on this matter. I will let Ian decide what to do here.

Alex

networkimprov

comment created time in 3 months

issue commentgolang/go

Is anybody working on the alternative approach mentioned above?

No, I am not making this change.

@zx2c4 said in https://github.com/golang/go/issues/31528#issuecomment-524641678

It's impossible to implement WireGuard securely if timers don't take into account sleep time.

CL 191957 fixes this problem. If we revert the CL and replace it with QueryUnbiasedInterruptTime, the problem will reappear.

Also using QueryUnbiasedInterruptTime will make nanotime implementation about 2 times slower. See https://github.com/golang/go/issues/31528#issuecomment-500270601 for details.

Alex

networkimprov

comment created time in 3 months

issue commentgolang/go

However, if you assumed that O_SYNC worked on your platform (which wouldn't be unreasonable on Windows given the existence of FILE_FLAG_WRITE_THROUGH) and turned out to be incorrect you could experience unexpected data loss (as in dgraph-io/badger#1084).

I don't see any promise of data loss prevention in O_SYNC documentation.

It says

O_SYNC int = syscall.O_SYNC // open for synchronous I/O.

whatever that means.

Alex

aschmahmann

comment created time in 3 months

issue commentgolang/go

Alex, did you see #35150 (comment)?

Yes, I did.

Alex

networkimprov

comment created time in 3 months

issue commentgolang/go

It isn't clear to me if this is something that the standard library itself should handle, but I recently wrote some Go code to do just this if it helps: https://github.com/WireGuard/wgctrl-go/blob/master/internal/wguser/conn_windows.go#L196.

@mdlayher the code you are referring to, implements functionality to connect to Windows named pipe. What we want in this issue is to list all files under \\.\pipe. Not the same thing.

The same documentation suggests to use NtQueryDirectoryFile of library NtosKrnlwhich is unavailable in the syscallpackage. ... Adding the relevant call require seems more like a proposal.

@iWdGo using NtQueryDirectoryFile sounds like it might work. And you don't need to add NtQueryDirectoryFile to syscall package. We can put it in internal/syscall/windows if it works.

Alex

Codehardt

comment created time in 3 months

issue commentgolang/go

I still do not see any problem here.

Sorry.

Alex

networkimprov

comment created time in 3 months

issue commentgolang/go

Thank you for creating this issue.

I am not an expert here. But, I am not against mapping syscall.O_SYNC onto FILE_FLAG_WRITE_THROUGH.

Alex

aschmahmann

comment created time in 3 months

issue commentgolang/go

Alex

tamayika

comment created time in 3 months

issue commentgolang/go

@apmckinlay yes both / and \ should work in this situation.

Would you like to send a fix for this? This https://golang.org/doc/contribute.html is how to contribute.

Thank you.

Alex

apmckinlay

comment created time in 3 months

issue commentgolang/go

changing

(*[2048]uint16)(unsafe.Pointer(&buf.PathBuffer[0]))[:]


to

(*[2048]uint16)(unsafe.Pointer(&buf.PathBuffer[0]))[:buflen:buflen]


might do the trick

Thank you, Bryan. That does the trick.

I knew the trick, but I forgotten about it. Moving on for now.

@mdempsky I also had to change

--- a/src/internal/syscall/windows/reparse_windows.go
+++ b/src/internal/syscall/windows/reparse_windows.go
@@ -60,8 +60,9 @@ type SymbolicLinkReparseBuffer struct {

// Path returns path stored in rb.
func (rb *SymbolicLinkReparseBuffer) Path() string {
-       p := (*[0xffff]uint16)(unsafe.Pointer(&rb.PathBuffer[0]))
-       return syscall.UTF16ToString(p[rb.SubstituteNameOffset/2 : (rb.SubstituteNameOffset+rb.SubstituteNameLength)/2])
+       n1 := rb.SubstituteNameOffset / 2
+       n2 := (rb.SubstituteNameOffset + rb.SubstituteNameLength) / 2
+       return syscall.UTF16ToString((*[0xffff]uint16)(unsafe.Pointer(&rb.PathBuffer[0]))[n1:n2:n2])
}

type MountPointReparseBuffer struct {
@@ -83,6 +84,7 @@ type MountPointReparseBuffer struct {

// Path returns path stored in rb.
func (rb *MountPointReparseBuffer) Path() string {
-       p := (*[0xffff]uint16)(unsafe.Pointer(&rb.PathBuffer[0]))
-       return syscall.UTF16ToString(p[rb.SubstituteNameOffset/2 : (rb.SubstituteNameOffset+rb.SubstituteNameLength)/2])
+       n1 := rb.SubstituteNameOffset / 2
+       n2 := (rb.SubstituteNameOffset + rb.SubstituteNameLength) / 2
+       return syscall.UTF16ToString((*[0xffff]uint16)(unsafe.Pointer(&rb.PathBuffer[0]))[n1:n2:n2])
}


because go test crashed. But these weren't listed at https://github.com/golang/go/issues/34972#issuecomment-544774978 I am worried we might miss more like it. Should I try different grep '\*\[.*unsafe.*:' expression?

Thank you.

Alex

mdempsky

comment created time in 3 months

issue commentgolang/go

We can look into whether this could be supported in a future version of Windows containers.

Sounds good.

I think it would be reasonable to ignore failure in this case.

Do you suggest we always ignore PowerRegisterSuspendResumeNotification failure? Or you suggest to ignore it on particular Windows version? How do I decide when to ignore PowerRegisterSuspendResumeNotification failure?

The bigger question to me is whether the change that introduced this regression was the right one at all. The Windows kernel team changed timer behavior in Windows 8 to stop advancing relative timeouts on wake. Otherwise when you open your laptop lid, every timer in the system goes off all at once and you get a bunch of unpredictable errors.

Sounds reasonable to me. But you could also see problems with your design decision too - see, for example #31528. It could be confusing when your timer takes longer after you have laptop lid closed.

I will let Ian and @zx2c4 fight that fight with you.

Alex

KnicKnic

comment created time in 3 months

issue commentgolang/go

My understanding is package syscall is frozen, and no new functions are to be added.

This seems like maybe it would be a reasonable exception?

I had to use new function in syscall package. So it cannot be in internal/syscall/windows, because internal/syscall/windows already imports syscall. I will put in syscall for now, and Ian could decide.

I went up to

os/os_windows_test.go: copy((*[2048]uint16)(unsafe.Pointer(&buf.PathBuffer[0]))[:], target.pathBuf)

now. go test crashes with

panic: runtime error: unsafe pointer conversion [recovered]
panic: runtime error: unsafe pointer conversion

goroutine 138 [running]:
testing.tRunner.func1(0xc00028e120)
/home/a/go/src/testing/testing.go:916 +0x60b
panic(0x5afc80, 0xc000328680)
/home/a/go/src/runtime/panic.go:961 +0x16b
os_test.createMountPoint(0xc0003394f0, 0x44, 0xc0002a5c10, 0xc00074007e, 0x3a)
/home/a/go/src/os/os_windows_test.go:266 +0x119
os_test.TestDirectoryJunction.func1(0xc0003394f0, 0x44, 0xc00029a180, 0x3a, 0x44, 0xc0000861d0)
/home/a/go/src/os/os_windows_test.go:285 +0xf5
/home/a/go/src/os/os_windows_test.go:123 +0x671
os_test.TestDirectoryJunction(0xc00028e120)
/home/a/go/src/os/os_windows_test.go:317 +0x28c
testing.tRunner(0xc00028e120, 0x5d9eb0)
/home/a/go/src/testing/testing.go:954 +0xe3
created by testing.(*T).Run
/home/a/go/src/testing/testing.go:1005 +0x35e


so it is definitely a problem. What is happening here, we have

package windows // import "internal/syscall/windows"

type MountPointReparseBuffer struct {
// The integer that contains the offset, in bytes,
// of the substitute name string in the PathBuffer array,
// computed as an offset from byte 0 of PathBuffer. Note that
// this offset must be divided by 2 to get the array index.
SubstituteNameOffset uint16
// The integer that contains the length, in bytes, of the
// substitute name string. If this string is null-terminated,
// SubstituteNameLength does not include the Unicode null character.
SubstituteNameLength uint16
// PrintNameOffset is similar to SubstituteNameOffset.
PrintNameOffset uint16
// PrintNameLength is similar to SubstituteNameLength.
PrintNameLength uint16
PathBuffer      [1]uint16
}


but PathBuffer is longer than one uint16 long - it is PrintNameLength elements long.

See MountPointReparseBuffer union part of REPARSE_DATA_BUFFER

https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/ns-ntifs-_reparse_data_buffer

Windows reads or writes uint16 string starting from PathBuffer[1]. How do I write uint16 string there?

That is what we do at this moment.

https://github.com/golang/go/blob/e6fb39aa6869fa5033b87c14a9826598679cb47d/src/os/os_windows_test.go#L266

Thank you.

Alex

mdempsky

comment created time in 3 months

issue commentgolang/go

@KnicKnic

Thank you for report.

We should at least add Windows error number to the panic message. Lucky we have you, who worked out what the error number is.

The problem is that the PowerRegisterSuspendResumeNotification returns error 2 - ERROR_FILE_NOT_FOUND

This error looks strange. Maybe PowerRegisterSuspendResumeNotification is broken when running in docker container. Unfortunetly I don't have docker to verify the error.

@jstarks should PowerRegisterSuspendResumeNotification work in container? Thank you.

Alex

KnicKnic

comment created time in 3 months

issue commentgolang/go

func utf16PtrToString(p *uint16) string {


I think this function will be useful to make other code '-gcflags=all=-d=checkptr' safe.

I could add this function to internal/syscall/windows package. But maybe we should add it to syscall package instead. We could recommend the function to be used in external code instead of syscall.UTF16ToString.

What do you think? Is it too earlier to make this decision?

Thank you.

Alex

mdempsky

comment created time in 3 months

issue commentgolang/go

I'll start writing a patch, but I'll have to wait till tomorrow (East Aus Time) for my employer to sign the CLA.

No worries. When you are ready.

Alex

LiamHaworth

comment created time in 3 months

issue commentgolang/go

Right now, at least from my perspective, there isn't really an easy way to automate testing of this code but I did provide a gist in a previous comment with code that replicates the issue.

Your Gist uses Windows service. And not a real service. Don't worry about the test, if you cannot come up with something simple.

What is the reasoning behind this? From what I understood of the code base, the internal package can be accessed by only packages in the standard library (e.g. os/exec)

The problem with your code fragment is that it uses internal/syscall/windows package, and the package contains no non-windows code. So, if you add EnvironForSysProcAttr there, you would have to implement 2 different versions: windows and non-windows. And that would be unusual to have non-windows code in this package.

The windows.EnvironForSysProcAttr name should give you a clue. This name starts with windows, but the code is for non-windows. I think it would be confusing for future users.

I think the choice is either to implement windows specific internal/syscall/windows.EnvironForSysProcAttr and use it in both os and os/exec. Or create all platform supported version in internal/syscall.EnvironForSysProcAttr and use it in os and os/exec.

Alex

LiamHaworth

comment created time in 3 months

issue commentgolang/go

This is now implemented at head. Is there anything remaining to be done for this issue?

No. All looks good here. Thank you very much.

Alex

comment created time in 3 months

issue commentgolang/go

@alexbrainman and @mattn, do you think we should just hard-code FILE_SHARE_DELETE in these ephemeral, non-os.Open uses of syscall.CreateFile?

What is the problem? How do I reproduce the problem?

Thank you.

Alex

networkimprov

comment created time in 3 months

issue commentgolang/go

I was thinking that only windows specific code would call environForSysProcAttr. We can add to internal/syscall if that turns out to be cleaner.

Sure that is fine with me too. Except @LiamHaworth code fragment from https://github.com/golang/go/issues/35314#issuecomment-549694390 cannot refer to internal/syscall/windows package directly. But I am fine either way, whatever is simpler.

@LiamHaworth let me know, if you are still want to send a fix for this.

Thank you.

Alex

LiamHaworth

comment created time in 3 months

issue commentgolang/go

But in the calling method (os/exec/exec.go:416) the Env variable it set by

Yes. We have forgotten about os/exec package.

it doesn't look like there was any testing.

Do you have suggestion for a test?

Also happy to take on responsibility to submit a fix, just want suggestions on the best way to do so

Sure, that would be nice. Here https://golang.org/doc/contribute.html is how to contribute.

we should move the Windows version of environForSysProcAttr to internal/syscall/windows

@ianlancetaylor internal/syscall/windows package only contains windows specific code at this moment. As I understand your proposal, there will be all OSes version of environForSysProcAttr there. Should we add this function to internal/syscall instead?

Thank you.

Alex

LiamHaworth

comment created time in 3 months

issue commentgolang/go

@mdempsky I need more help.

I am trying to convert this line

net/interface_windows.go: Name: syscall.UTF16ToString((([10000]uint16)(unsafe.Pointer(aa.FriendlyName)))[:]),

https://github.com/golang/go/blob/3873e5497d21979910a1ec7cf90a34577fa1f6ae/src/net/interface_windows.go#L61

But FriendlyName is 0 terminated uint16 string (see https://docs.microsoft.com/en-us/windows/win32/api/iptypes/ns-iptypes-ip_adapter_addresses_lh for the reference). There is no length of FriendlyName provided anywhere. What do we do here?

The test is also crashes

:\>u:\test -test.short
panic: runtime error: unsafe pointer conversion

goroutine 1 [running]:
net.interfaceTable(0x0, 0xa80, 0xa80, 0x66e940, 0x1, 0xc000087ae0)
/home/a/go/src/net/interface_windows.go:61 +0xec
net.Interfaces(0x20, 0x28, 0xc0000667e0, 0x47fbb1, 0x890854)
/home/a/go/src/net/interface.go:100 +0x35
net.loopbackInterface(0x65d0c0)
/home/a/go/src/net/interface_test.go:20 +0x3b
net.setupTestData()
/home/a/go/src/net/main_test.go:102 +0x5c
net.TestMain(0xc0000ac000)
/home/a/go/src/net/main_test.go:49 +0x2d
main.main()
_testmain.go:474 +0x13c

C:\>


Thank you.

Alex

mdempsky

comment created time in 4 months

issue commentgolang/go

The way the check currently works is

Thank you for taking time to explain.

I think we do need to change crypto/x509 package.

Alex

mdempsky

comment created time in 4 months

issue commentgolang/go

Does anyone object to changing the default buildmode for Windows to pie in Go 1.15? (It seems too late in the cycle for Go 1.14.)

How about we will make -buildmode=pie work on Windows (issue #27144) for Go 1.14? Then in Go 1.14 release notes we will announce the change and declare that -buildmode=pie will become default in Go 1.15. Gives everyone plenty of time to prepare for the change.

If this suggestion is accepted, I ask for dispensation to implement #27144 after Go 1.14 freeze. It should not be hard to implement, but there are only few days left.

Alex

zx2c4

comment created time in 4 months

issue commentgolang/go

crypto/x509/root_windows.go: simpleChains := ([1 << 20]syscall.CertSimpleChain)(unsafe.Pointer(simpleChain))[:] crypto/x509/root_windows.go: elements := ([1 << 20]syscall.CertChainElement)(unsafe.Pointer(lastChain.Elements))[:] crypto/x509/root_windows.go: encodedCert := ([1 << 20]byte)(unsafe.Pointer(cert.EncodedCert))[:] crypto/x509/root_windows.go: buf := ([1 << 20]byte)(unsafe.Pointer(cert.EncodedCert))[:]

@mdempsky checker does not complain about crypto/x509 code

c:\>go test -a -short -gcflags=all=-d=checkptr crypto/x509
ok      crypto/x509     0.393s


Why? All 4 lines deals with external to Go memory returned by Windows API. Is that why?

Thank you.

Alex

mdempsky

comment created time in 4 months

issue commentgolang/go

I am not convinced by the idea of overloading -buildmode with fundamental security features.

I am not married to making -buildmode=pie default idea. If you have better suggestion I am fine with that. -buildmode=pie will not require us to add any new flags.

I think we should just turn on ASLR everywhere, becoming consistent with arm.

I doubt we have many windows-arm Go users. We don't even have windows-arm builder. So we should treat ASLR as completely new feature.

I haven't checked the state of debug symbols on windows with buildmode=pie.

I agree, we should make sure Delve works with ASLR enabled windows binaries. We might discover that "build non ASLR executable" flag will get handy with Delve. Thank you @aarzilli for reminding us about Delve.

Let's please take the conservative approach here. We know how to generate a non-PIE. If we change the default to be PIE, I see no reason at all to make it hard to generate a non-PIE.

I completely agree.

Alex

zx2c4

comment created time in 4 months

issue commentgolang/go

I would also like to add that this proposal assumes that #27144 is implemented as -buildmode=pie. So this current proposal is effectively means that -buildmode=pie will become default on Windows (as suggested by @ianlancetaylor at https://github.com/golang/go/issues/27144#issuecomment-546633412).

Alex

zx2c4

comment created time in 4 months

issue commentgolang/go

Does anybody think it would be a bad idea to change the default to be PIE on Windows?

I think we should create new proposal for that, and see what happens.

The only downside I see, we might discover some bugs.

Does anybody have a suggestion as to how it should work? Would we keep the current -buildmode options and change the default for a main package to be -buildmode=pie, while -buildmode=exe would continue to build a non-PIE?

That sounds good to me. As long as -buildmode=pie is the default, and users don't need to type it, I think we will be alright.

ARM does it unconditionally. What if we just did it that way?

Or do you think providing the option is important?

I have an impression that we don't have choice for ARM.

And I think it is important, that we have easy way to disable ASLR if required. I would use that myself while debugging. And, if we follow Ian's suggestion, it is just -buildmode=exe option away.

Alex

comment created time in 4 months

issue commentgolang/go

So if the fix was reverted,

@jakirkham

Change https://golang.org/cl/203603 has not been submitted yet. So the fix is still there.

We are considering to use this copy of Mingw - http://musl.cc/x86_64-w64-mingw32-native.zip (see https://github.com/golang/go/issues/35006#issuecomment-546573751 for details). If you are affected by this current issue, can you try this Mingw to see, if it fixes your problem?

Thank you.

Alex

mattn

comment created time in 4 months

issue commentgolang/go

This is what all Go users use. Including our builders (see https://github.com/golang/go/wiki/WindowsBuild ).

If we are to change this, then we should all agree what Mingw we use. Perhaps we should even provide the url in https://golang.org/doc/install - if there are many Mingw versions and they work differently, then we should be explicit there. And we should update our builders to run the same software.

I am fine using http://musl.cc if everyone agrees. @rsc and @ianlancetaylor ?

And we won't be fixing issues like #23649, because they are not reproducible with http://musl.cc . @mattn can you, please, verify? I used http://musl.cc/x86_64-w64-mingw32-native.zip

I can't explain why the other toolchain causes the error,

@zv-io thank you for taking time to comment here. The fix for #23649 is here

And the gist of the fix is, it works around commit b295099 from git://git.code.sf.net/p/mingw-w64/mingw-w64 Does your Mingw build includes this commit? The commit (as I understand it) adds __imp___acrt_iob_func symbol. That breaks Go internal linker, because Go assumes that all symbols starting with __imp_ live in external DLLs. The bug does not happens with http://musl.cc - I did not investigate it enough to understand why.

Alex

alexbrainman

comment created time in 4 months

issue commentgolang/go

If GCC doesn't generate PIE by default, it's not obvious to me that go should.

I don't think you should dismiss the fact that Microsoft compilers do generate PIE by default. Most Windows executables are built by Microsoft tools.

Clearly we should make sure that go build -buildmode=pie builds a PIE.

Sure, we could do that. But I don't see why Go developers would use this command option. ASLR is supposed to help make Windows more secure. Developers don't care about their users OS security. It is OS admin responsibility to make system secure. Developers will use whatever default build mode is.

Also current windows-arm version of Go already uses PIE by default

I think our current go command flags are just confusing in that area.

Alex

comment created time in 4 months

issue commentgolang/go

We should direct users to using actually maintained compilers.

I am all for making better tools available to Go users.

But we still need to fix bugs that Go users experience while using whatever tools they use. How do you propose we handle bugs like #23649 if we don't use same compilers as #23649 users?

Alex

alexbrainman

comment created time in 4 months

issue commentgolang/go

What do other compilers do? In particular, what do GCC and clang do?

I do not know about clang. But I tried Mingw GCC C, and Microsoft C and C#.

According to https://stackoverflow.com/questions/8554014/how-to-know-whether-a-dll-uses-aslr-or-not

GCC does not produce ASLR binaries by default:

C:\>dumpbin /headers mingw.exe | sed "/OPTIONAL HEADER VALUES/,+30!d"
20B magic # (PE32+)
1E00 size of code
3800 size of initialized data
A00 size of uninitialized data
14E0 entry point (00000000004014E0)
1000 base of code
400000 image base (0000000000400000 to 0000000000411FFF)
1000 section alignment
200 file alignment
4.00 operating system version
0.00 image version
5.02 subsystem version
0 Win32 version
12000 size of image
1B418 checksum
3 subsystem (Windows CUI)
0 DLL characteristics
200000 size of stack reserve
1000 size of stack commit
100000 size of heap reserve
1000 size of heap commit
10 number of directories
0 [ 0] RVA [size] of Export Directory
8000 [ 7C4] RVA [size] of Import Directory
0 [ 0] RVA [size] of Resource Directory
5000 [ 27C] RVA [size] of Exception Directory
0 [ 0] RVA [size] of Certificates Directory

C:\>


But both Microsoft C and C# both generate ASLR by default:

C:\>dumpbin /headers ms_c.exe | sed "/OPTIONAL HEADER VALUES/,+30!d"
20B magic # (PE32+)
13200 size of code
D200 size of initialized data
0 size of uninitialized data
1388 entry point (0000000140001388)
1000 base of code
140000000 image base (0000000140000000 to 0000000140023FFF)
1000 section alignment
200 file alignment
6.00 operating system version
0.00 image version
6.00 subsystem version
0 Win32 version
24000 size of image
0 checksum
3 subsystem (Windows CUI)
8160 DLL characteristics
Dynamic base
NX compatible
Terminal Server Aware
100000 size of stack reserve
1000 size of stack commit
100000 size of heap reserve
1000 size of heap commit
10 number of directories
0 [ 0] RVA [size] of Export Directory

10B magic # (PE32)
600 size of code
800 size of initialized data
0 size of uninitialized data
24DE entry point (004024DE)
2000 base of code
4000 base of data
400000 image base (00400000 to 00407FFF)
2000 section alignment
200 file alignment
4.00 operating system version
0.00 image version
4.00 subsystem version
0 Win32 version
8000 size of image
0 checksum
3 subsystem (Windows CUI)
8540 DLL characteristics
Dynamic base
NX compatible
No structured exception handler
Terminal Server Aware
100000 size of stack reserve
1000 size of stack commit
100000 size of heap reserve
1000 size of heap commit
10 number of directories

C:\>


Alex

comment created time in 4 months

issue commentgolang/go

I'd recommend using the ones from https://musl.cc/ . They're up to date and work very well:

http://musl.cc/x86_64-w64-mingw32-native.zip http://musl.cc/i686-w64-mingw32-native.zip

I tried running all.bat on the current tip against

http://musl.cc/x86_64-w64-mingw32-native.zip

and it all passes. While, if I do the same against

https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/8.1.0/threads-win32/seh/x86_64-8.1.0-release-win32-seh-rt_v6-rev0.7z

it fails in debug/pe with error similar to https://github.com/golang/go/issues/23649#issuecomment-487344450

Why is that?

And, while I, probably, prefer your version of gcc, our builders should test against what most users use.

Alex

alexbrainman

comment created time in 4 months

issue commentalexbrainman/odbc

Can I force in may golang code the support Unicode?

I doubt.

I think your unixODBC driver or configuration is the problem. I suspect, if you could make iusql command to work, your Go program will just work. See #71 for something similar.

Alex

comment created time in 4 months

issue commentgolang/go

Here's the root cause for the integer size bug:

They're writing an integer of size 8 as type REG_DWORD, which REG_DWORD requires an integer of size 4. The key is called "lParam", which means I bet this was originally of type LPARAM which is of type LONG_PTR which is the size of a pointer. So probably the original C code was something like:

RegSetValueExW(hKey, L"lParam", 0, REG_DWORD, (const BYTE *)&whatever->lParam, sizeof(whatever->lParam));


This worked fine in the 32-bit Windows days, and then broke when they recompiled that binary for 64-bit.

I only know how to report security bugs to MSRC, but not other bugs to whatever channels Microsoft has for those. If somebody wants to direct a Microsoft person toward this thread, maybe this can get fixed.

@jstarks maybe you could report this bug at the right channel?

Thank you.

Alex

comment created time in 4 months

issue commentgolang/go

I'm in way over my head,

Same here. I will let Jason reply to you, if he wants to. I think we should leave things as they are.

Alex

zx2c4

comment created time in 4 months

more