profile
viewpoint

randall77/hprof 71

Heap profile reader for Go

randall77/heapdump14 25

Heap dump reader & visualizer for Go 1.4

randall77/corelib 12

A tool for reading Go core files

randall77/factorlib 5

A library for factoring big integers in Go

randall77/compilercover 4

Tool to generate coverage reports on the Go compiler

randall77/memmove 3

Test for memmove speed

randall77/rulecover 3

Tool for producing a cover profile of SSA .rules files

randall77/prominence 2

A program to compute the prominence of mountains

randall77/2016-talks 1

Slides and Links to slides for 2016 talks

randall77/errors 1

Simple error handling primitives

IssuesEvent

issue commentgolang/go

cmd/compile: constant propagation in compiler converts signaling NaN to quiet NaN

CL reverted, reopening.

fxamacker

comment created time in 4 hours

issue closedgolang/go

runtime: golang 1.14.rc1 3-5% performance regression from golang 1.13 during protobuf marshalling [1.12 backport]

@randall77 requested issue #37086 to be considered for backport to the next 1.12 minor release.

@gopherbot please open a backport issue.

I don't think this performance regression qualifies for a backport, but I'll let the release guys make that call.

closed time in a day

gopherbot

issue commentgolang/go

runtime: golang 1.14.rc1 3-5% performance regression from golang 1.13 during protobuf marshalling [1.12 backport]

Don't need to go back to 1.12, the performance regression was introduced in 1.13.

gopherbot

comment created time in a day

issue commentgolang/go

runtime: golang 1.14.rc1 3-5% performance regression from golang 1.13 during protobuf marshalling

@gopherbot please open a backport issue.

I don't think this performance regression qualifies for a backport, but I'll let the release guys make that call.

howardjohn

comment created time in a day

issue commentgolang/go

cmd/compile: partial cse of phi inputs?

Sounds reasonable. So we look for (Phi (Op x a) (Op y a) (Op z a)) and replace with (Op (Phi x y z) a)

That seems like it would almost always be a win. Maybe it isn't if the (Op * a) have other uses besides this Phi. Probably not a big deal, but we could condition on the Uses count.

josharian

comment created time in a day

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

Looks like this was fixed by an update on Corellium's part on our about 2/17.

bcmills

comment created time in a day

issue openedgolang/go

runtime: using fallback hash unconditionally

First reported in #37086 by @Slinso:

i am not sure if this is related, but i saw some differences in map access performance, too. In my case I am not using interface{} as a key, but instead a map[int]int

https://github.com/SlinSo/random/blob/master/mapaccess_test.go

pprof excerpt: go1.13.8: 0.73s 9.53% 82.64% 0.73s 9.53% runtime.aeshash64

go1.14rc1: 1230ms 15.81% 66.20% 1410ms 18.12% runtime.memhash64Fallback 140ms 1.80% 97.43% 140ms 1.80% runtime.memhash64

We should not be using the fallback hash unless the processor can't do the AES instructions (which, as evidenced by the 1.13 results, it has).

created time in 12 days

issue commentgolang/go

runtime: golang 1.14.rc1 3-5% performance regression from golang 1.13 during protobuf marshalling

(Unrelated to this issue, though.)

howardjohn

comment created time in 12 days

issue commentgolang/go

runtime: golang 1.14.rc1 3-5% performance regression from golang 1.13 during protobuf marshalling

That's a bug. Yikes! Fix coming.

howardjohn

comment created time in 12 days

issue commentgolang/go

compile/arm64: go test fail for arm64

I guess you want to build go1.13.7 with go1.8.7 ? I tried this process but all tests passed. In addition, you should set GOROOT_BOOTSTRAP instead of GO_BOOTSTRAP.

Right.

You also need to set go1.8.7 as GOROOT and add it to PATH.

That should not be necessary.

darmac

comment created time in 14 days

issue commentgolang/go

compile/arm64: go test fail for arm64

git clone https://github.com/golang/go.git

In which directory did you run this? If it was /home/xiaojun, then you just installed the two Go releases on top of each other. That's not going to work. ...but maybe not? I would be surprised that it got this far if that's what you did.

darmac

comment created time in 14 days

issue commentgolang/go

os: small-count Readdirnames followed by seek-to-zero can lead to duplicate names

Yes, looks like we need a similar fix to CL 209961 for non-darwin unix. CL coming. It's been broken since at least 1.11, so this will wait until 1.15.

chris3torek

comment created time in 14 days

issue commentgolang/go

runtime: Darwin slow when using signals + int->float instructions

I reported a bug to apple here. We'll see if we get any traction on that.

randall77

comment created time in 14 days

issue commentgolang/go

runtime: Darwin slow when using signals + int->float instructions

On our 1.11 builders, my C repro is ~14% slower with the VZEROUPPER commented out. On 1.12, ~13% slower. On 1.14, ~11% faster. On 1.15, ~11% faster.

So for some reason this bug doesn't appear on the builders like it does on my desktop. I could imagine that some virtualization layer hides the bug, and the VZEROUPPER instruction has a cost.

randall77

comment created time in 14 days

issue commentgolang/go

runtime: Darwin slow when using signals + int->float instructions

Here's a C/assembly reproducer:

main.c

#include <stdio.h>
#include <stdint.h>
#include <signal.h>

#define N 64
#define SIG SIGFPE

// Does z = y+float64(x) for 64 entries. This is the main benchmark.
void add(uint64_t *x, double *y, double *z);

uint64_t x[N];
double y[N];
double z[N];

void handler(int sig, siginfo_t *info, void *arg) {
}

int main(int argc, char *argv[]) {
  // Install handler for a signal.
  // sa_sigaction is required; sa_handler doesn't trigger the bug.
  struct sigaction act;
  act.sa_sigaction = handler;
  act.sa_flags |= SA_SIGINFO;
  sigaction(SIG, &act, NULL);
  // Raise the signal, which dirties the upper bits of YMM registers.
  raise(SIG);

  // Run the benchmark.
  for (int i = 0; i < 40000000; i++) {
    add(&x[0], &y[0], &z[0]);
  }
  return 0;
}

add.s:

	.globl	_add
_add:
	// Uncomment the vzeroupper to "fix" the bug.
	//vzeroupper
	movq	$0, %rax
loop:
	cmpq	$64, %rax
	je	done
	movq	(%rdi, %rax, 8), %rcx
	movq	%rcx, %xmm0
	addsd	(%rsi, %rax, 8), %xmm0
	movsd	%xmm0, (%rdx, %rax, 8)
	incq	%rax
	jmp	loop

done:
	ret
randall77

comment created time in 14 days

issue commentgolang/go

runtime: Darwin slow when using signals + int->float instructions

Can we put the VZEROUPPER in our signal handler return path, when the code that was interrupted was Go code? That would be a more efficient way to ensure avx-cleanliness of float registers for Go.

randall77

comment created time in 14 days

issue commentgolang/go

runtime: go1.14rc1 performance regression

I've split out the Darwin-specific weirdness into #37174 .

klauspost

comment created time in 14 days

issue openedgolang/go

runtime: Darwin slow when using signals + int->float instructions

Split off from #37121

package bench1

import (
	"math"
	"testing"
)

const N = 64

func BenchmarkFast(b *testing.B) {
	var x, y, z [N]float32

	for i := 0; i < b.N; i++ {
		mulFast(&x, &y, &z)
	}
}

func mulFast(x, y, z *[N]float32) {
	for i := 0; i < N; i++ {
		z[i] = x[i] * y[i]
	}
}

func BenchmarkSlow(b *testing.B) {
	var z [N]float32
	var x, y [N]uint32

	for i := 0; i < b.N; i++ {
		mulSlow(&x, &y, &z)
	}
}

func mulSlow(x, y *[N]uint32, z *[N]float32) {
	for i := 0; i < N; i++ {
		z[i] = math.Float32frombits(x[i]) * math.Float32frombits(y[i])
	}
}
% ~/go1.12.9/bin/go test bench1_test.go -test.bench .\* -test.benchtime=10000000x 
goos: darwin
goarch: amd64
BenchmarkFast-16    	10000000	        55.9 ns/op
BenchmarkSlow-16    	10000000	        61.1 ns/op
PASS
% ~/go1.12.9/bin/go test bench1_test.go -test.bench .\* -test.benchtime=10000000x -test.cpuprofile=cpu.prof
goos: darwin
goarch: amd64
BenchmarkFast-16    	10000000	        89.7 ns/op
BenchmarkSlow-16    	10000000	       223 ns/op
PASS

For some strange reason, code that includes int->float instructions runs a lot slower when profiling is on.

This bug is reproducible from at least 1.11.

created time in 14 days

issue commentgolang/go

runtime: go1.14rc1 performance regression

Re the large slowdown on Darwin: there's something strange going on with signals here. If I build this benchmark with 1.13.6, I get:

% ./113.test -test.bench .\* -test.benchtime=1000000x                          
Benchmark_tokens_EstimatedBits-16    	 1000000	       657 ns/op	   1.52 MB/s
% ./113.test -test.bench .\* -test.benchtime=1000000x  -test.cpuprofile=113.prof
Benchmark_tokens_EstimatedBits-16    	 1000000	      2551 ns/op	   0.39 MB/s

i.e., just turning on the profiler leads to an almost 4x slowdown.

1.14 is different because it uses signals all the time. So we always see the slow behavior unless both the profiler and the async preempt are off:

% ./tip.test -test.bench .\* -test.benchtime=1000000x                           
Benchmark_tokens_EstimatedBits-16    	 1000000	      2498 ns/op	   0.40 MB/s
% ./tip.test -test.bench .\* -test.benchtime=1000000x -test.cpuprofile=tip.prof
Benchmark_tokens_EstimatedBits-16    	 1000000	      2536 ns/op	   0.39 MB/s
% GODEBUG=asyncpreemptoff=1 ./tip.test -test.bench .\* -test.benchtime=1000000x -test.cpuprofile=tip.prof
Benchmark_tokens_EstimatedBits-16    	 1000000	      2530 ns/op	   0.40 MB/s
% GODEBUG=asyncpreemptoff=1 ./tip.test -test.bench .\* -test.benchtime=1000000x                          
Benchmark_tokens_EstimatedBits-16    	 1000000	       680 ns/op	   1.47 MB/s

Not sure what's going on here yet, but something about Darwin and signals is bad. This doesn't seem to happen for other benchmarks, though. Something about this particular benchmark is triggering the bad behavior.

So this really isn't new with 1.14. It just appears more often due to more use of signals.

klauspost

comment created time in 14 days

issue commentgolang/go

Proposal: Add `??` operator to select first non-zero value

None of your examples require the short-circuit behavior. I.e., first(ts ...T) T would work fine for them.

carlmjohnson

comment created time in 14 days

issue commentgolang/go

runtime: concurrent object placement on heap causes Linux kernel contention

@networkimprov We could add this as a benchmark. Feel free to submit a CL.

I don't think we want to make it a test. We abhor flaky tests, and I think this one would be hard to make reliable enough to include by default.

midom

comment created time in 16 days

issue commentgolang/go

runtime: 10ms-26ms latency from GC in go1.14rc1, possibly due to 'GC (idle)' work

On Sun, Feb 9, 2020 at 2:38 PM Richard L. Hudson notifications@github.com wrote:

The following may be relevant. From https://golang.org/pkg/runtime/ func GC https://golang.org/src/runtime/mgc.go?s=37860:37869#L1026

func GC()

GC runs a garbage collection and blocks the caller until the garbage collection is complete. It may also block the entire program.

In the section about the trace line:

If the line ends with "(forced)", this GC was forced by a runtime.GC() call.

The trace contains

gc 15 @432.576s 4%: 0.073+17356+0.021 ms clock, 0.29+0.23/17355/34654+0.084 ms cpu, 8697->8714->8463 MB, 16927 MB goal, 4 P GC forced

at the point indicated as the cause of the issue and the second screenshot looks like a call to runtime.GC is made and the mutator thread is stopped and the GC is run.

I would look into whether runtime.GC is called and if the call is in the dynamic scope of code being timed. If so remove the runtime.GC call and see if that makes any difference.

And kudos for helping out testing the new release candidate.

On Sun, Feb 9, 2020 at 2:03 PM Heschi Kreinick notifications@github.com wrote:

I think this is working as expected. At the heart of github.com/golang/groupcache/lru is a single large map < https://github.com/golang/groupcache/blob/8c9f03a8e57eb486e42badaed3fb287da51807ba/lru/lru.go#L33 . AFAIK the map has to be marked all in one go, which will take a long time.

This is no longer true. We now break large objects, like maps, into bounded-size chunks. https://go-review.googlesource.com/c/go/+/23540 That was submitted mid 2016, so Go1.8?

In the execution trace above you can see a MARK ASSIST segment below G21. If you click on that I think you'll find that it spent all of its time marking a single object, presumably the map. Austin and others will know better, of course.

Since a real server would need to lock the map, sharding it would be a good idea, which would also reduce the latency impact of a mark.

(cc @rsc https://github.com/rsc, since I think @cagedmantis https://github.com/cagedmantis made a typo.)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/golang/go/issues/37116?email_source=notifications&email_token=AAHNNHYCOXRMJYP62MDEGO3RCBHPVA5CNFSM4KRSQU32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELGU7RA#issuecomment-583880644 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AAHNNH2BVNJP6PVXBKZJ3ZTRCBHPVANCNFSM4KRSQU3Q

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/37116?email_source=notifications&email_token=ABUSAIHQ2VOL4AOHQ3P236LRCCAUTA5CNFSM4KRSQU32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELG2KAQ#issuecomment-583902466, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUSAIAKOYXJ4SA7QAJ23VDRCCAUTANCNFSM4KRSQU3Q .

thepudds

comment created time in 16 days

issue commentgolang/go

runtime: golang 1.14.rc1 3-5% performance regression from golang 1.13 during protobuf marshalling

Certainly we could swap the order of the regular memory check and the kind switch, and add a kindPtr case.

howardjohn

comment created time in 19 days

issue commentgolang/go

math/bits: TestDiv32PanicOverflow, fatal error: unexpected signal during runtime execution

I can't reproduce on linux/amd64 at this commit.

There's definitely something strange here:

runtime.printstring(0x577800, 0x647f50)
	/home/u/go/src/runtime/print.go:242 +0x78 fp=0xc00002e700 sp=0xc00002e6b0 pc=0x433fa8
math/bits.Div32(...)
	/home/u/go/src/math/bits/bits.go:501

The length of that string looks very wrong.

Any chance you have two Go instances installed, and the system is getting confused? We've seen instances like this when the compiler comes from one release and the stdlib from another.

mewmew

comment created time in 20 days

issue commentgolang/go

Rate limiter frequency does not correspond to expected frequency

I can reproduce on Linux. Both 1.13.7 and tip.

I can not reproduce on Darwin - that can do ~96K/sec.

Increasing the burst size fixes this on Linux. Using 10 instead of 1 fixes the problem. At a rate of 100K/sec, you should probably allow bursts >1.

Not sure what the underlying problem is. Timer quantization, perhaps?

lebovski

comment created time in 20 days

issue commentgolang/go

cmd/compile/internal/ssa: test TestNexting failing

@dr2chase

cagedmantis

comment created time in 20 days

issue commentgolang/go

hash/maphash: Document whether used hash algorithm is stable across future versions

It now says:

// Each Seed value is local to a single process and cannot be serialized
// or otherwise recreated in a different process.

There's no way to transfer a seed between processes (including the same binary across time). So you can't observe a different algorithm because you can't use the same seed.

nightlyone

comment created time in 20 days

issue commentgolang/go

proposal: runtime: panicmem should expose the address it's panicking about.

We could do something about this. The types of runtime panics are explicitly unspecified: https://golang.org/ref/spec#Run_time_panics We can add a Addr() method to the value we panic with, so you can do

type addrable interface {
    Addr() uintptr
}
e := recover()
if a, ok := e.(addrable), ok {
   _ = a.Addr()
}

Should we include kind-of also? Not sure. We can rapidly reach implementation-specific areas, which I don't think we want to do.

seebs

comment created time in 21 days

issue commentgolang/go

runtime: performance degradation in go 1.12

Here's a simple reproducer. I can reproduce on Linux.

package main

import (
	"fmt"
	"sync"
	"time"
)

var sink []byte

func main() {
	start := time.Now()
	var wg sync.WaitGroup
	for i := 0; i < 10; i++ {
		wg.Add(1)
		go func() {
			for i := 0; i < 16000; i++ {
				sink = make([]byte, 64*1024)
			}
			wg.Done()
		}()
	}
	wg.Wait()
	fmt.Printf("time: %f\n", time.Since(start).Seconds())
}

This test is really allocation heavy. It's going to depend on the precise details of how allocation is done. I get these times:

go1.11.13: 0.689 sec
go1.12.16: 0.973 sec
go1.13.7: 1.151 sec
tip: 1.099 sec

I think most of the difference has to do with how big the heap grows. Here's my eyeballing the maxiumum RSS (adding lots of iterations to the innermost loop so I can watch top while it runs):

go1.11.13: 310MB
go1.12.16: 202MB
go1.13.7: 158MB
tip: 98MB

This test is stressing the allocator so much that it goes way beyond 200% of live heap data, which should only be 640KB or so. You'll see that we've made steady improvements on how much over we go. Of course, that means that we'll run garbage collections more often, which will hurt the run time.

I'm not sure there's any bug here. It just seems like a different speed/memory tradeoff. To actually demonstrate a regression, I think we'd need to hold heap size constant somehow.

@mknyszek

interviewQ

comment created time in 21 days

issue commentgolang/go

cmd/compile: it is not possible to prevent FMA with complex values

Try computing x*y (as complex128s) using:

func cmul(x, y complex128) complex128 {
	return complex(float64(real(x)*real(y))-float64(imag(x)*imag(y)), float64(real(x)*imag(y))+float64(imag(x)*real(y)))
}

That should inhibit fma computation of floating-point multiply.

kortschak

comment created time in 23 days

issue commentgolang/go

cmd/compile: it is not possible to prevent FMA with complex values

I think you're right that there's no way to turn this off for arm64. @agnivade, the GODEBUG setting should only be for x86 (and x86 only uses fma instructions for math.FMA).

There's nothing in the spec that specifies how complex arithmetic is calculated, so I don't think we're breaking anything.

I'm curious as to what is breaking as a result of this. It's not like you can control the rounding in a complex multiply regardless (there are 3 roundings in a non-fma complex multiply, and just 2 roundings in a fma-enabled complex multiply).

kortschak

comment created time in 24 days

issue commentgolang/go

Escape analysis on interface calls

There's lots of issues about resolving interfaces at compile time, so we don't have to allocate their receiver or args: #8618 #18822 #23676 #33160

It's an interesting idea to resolve this at run time instead. Generate code for both cases and decide which allocation path to use.

It can get tricky:

i.Call(&ctx)
j.Call(&ctx)

Then we can only allocate ctx on the stack if both i and j have their Call method marked as not escaping its first argument.

At the time we allocate ctx, we might not know yet which interface methods it might get passed to.

ctx := myCtx{}
... initialize ctx ...
f().Call(&ctx)

Until we can inspect f's return value, we don't know whether ctx escapes.

Sometimes interfaces are wrappers. For example, bufio.NewWriter. When we assign one of those to the io.Writer interface, do we mark its Write method as not escaping its argument? That's a dynamic property of the underlying io.Writer that was passed to NewWriter. How do we associate those things? (We could always punt and be conservative here, but it would be nice to handle.)

kirillx

comment created time in 24 days

issue commentgolang/go

cmd/compile: eliminate all (some?) convT2{I,E} calls

Yes, I think there's more to be done. The escape case (f in my example) seems nonoptimal. It makes a temporary, then passes the address of that temporary to convT2Enoptr. We could just call newobject directly, then initialize its contents in compiled code. I don't think it would be any bigger (for pointerless types, anyway).

randall77

comment created time in 25 days

issue commentgolang/go

runtime: allow map hashes with different tradeoffs

@networkimprov There is also the danger of writing a hash function and an equals function that aren't compatible (two equal values hash to different things). When that happens, it leads to very confusing behavior.

eric-s-raymond

comment created time in 25 days

issue commentgolang/go

Add `strings.Reverse()` and/or `sort.InvertSlice()`

@networkimprov This proposal was originally about strings.Reverse. If you think that's a bad idea and want to propose sort.InvertSlice instead, open a new proposal or convince the OP to repurpose this one.

I agree with @ALTree 's questions about strings.Reverse.

I think a slice reverser would be a reasonably useful function to have. It's not a hard function to write yourself. Show us cases where people have written and used it. I'll start, sort of. See where we reverse-and-copy a slice. We could do that loop with a reverse function pluscopy. I don't understand why this function would live in package sort. It has nothing to do with sorting.

gsbhasin123

comment created time in 25 days

issue closedgolang/go

Issue with type conversion from float64 to string

GO version- go1.13.1

when converting from float64 to string - GO is changing the value for Odd numbers.

SAMPLE CODE-

package main import ( "fmt" "strconv" ) func main() { var g float64 g = 12643612166373552 var ss string ss = strconv.FormatFloat(g, 'f', 0, 64) fmt.Println(ss)

var g1 float64
g1 = 12643612166373557
var ss1 string
ss1 = strconv.FormatFloat(g1, 'f', 0, 64)
fmt.Println(ss1)

}

OUTPUT- 12643612166373552 12643612166373556

EXPECTED OUTPUT- 12643612166373552 12643612166373557

closed time in a month

neetikh

issue commentgolang/go

Issue with type conversion from float64 to string

You can print out the representable float64 numbers in this range. Do at the end of your example:

	fmt.Println("\n")
	for i := 0; i < 4; i++ {
		fmt.Println(strconv.FormatFloat(g, 'f', 0, 64))
		g = math.Nextafter(g, 1e100)

You'll see that 12643612166373557 is not representable as a float64. It gets rounded to 12643612166373556.

neetikh

comment created time in a month

issue commentgolang/go

RFE: Allow map hashes with different tradeoffs

aeshash512

Where did 512 come from? That doesn't exist.

What do your keys look like (type, size, etc.)? Do you have a suggestion for an alternative, and how fast do you think it would be?

eric-s-raymond

comment created time in a month

issue commentgolang/go

proposal: Go 2: byte array <-> string conversions

let alone passing cap as a parameter to that function redundantly.

That is true, slicebytetostring never uses the capacity. I guess just fixing that is equivalent to my runtime.arraybytetostring proposal.

I thought Go compiler allocates space for string on stack, and just copies ptr and len.

It always has to do at least a copy of the data, because the compiler currently has no analysis to prove that the byte array is not modified subsequently (#31506, #2205).

If the result doesn't escape, it can allocate the backing array on the stack. But I don't think the details of that are much different in the two cases. Read runtime/string.go:slicebytetostring for the gory details.

jfcg

comment created time in a month

issue commentgolang/go

proposal: Go 2: byte array <-> string conversions

this will use extra 24 bytes on the stack compared to a byte array:

func f() { var buf = []byte("file000") ... }

Comparing, say,

func f() {
    ba := [3]byte{65, 65, 65}
    s := string(ba[:])
}

and

func g() {
    ba := [3]byte{65, 65, 65}
    s := string(ba) // proposed new feature
}

f has to call runtime.slicebytetostring which takes 24 bytes for the slice. g would have to call a hypothetical runtime.arraybytetostring which needs to take a pointer and a size, 16 bytes. In either case, this space is shared between all calls out of f (or g). So if you have a recursive call with 3 word-sized arguments (a string and an int, say), the conversion mentioned here costs no space at all. At worst it costs 8 bytes.

We could easily optimize the expression string(ba[:]) to only pass a ptr+len instead of a ptr+len+cap, and then it wouldn't cost any space, ever.

jfcg

comment created time in a month

issue closedgolang/go

type of math.MaxInt64 should be int64 instead of int

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

1.13 <pre> $ go version

</pre>

Does this issue reproduce with the latest release?

Yes, repro-ed on go playground

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env

</pre></details>

What did you do?

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. --> https://play.golang.org/p/LGRoTC_euz0

What did you expect to see?

Should compile,

What did you see instead?

Compiler complaining math.MaxInt64 is of type int.

closed time in a month

vitesse-ftian

issue commentgolang/go

type of math.MaxInt64 should be int64 instead of int

This is intended behavior.

math.MaxInt64 is an ideal constant. It does not have a type as such. Your imax variable gets the default type for integer constants, which is int. You should instead do either of these:

imax := int64(math.MaxInt64)
var imax int64 = math.MaxInt64

See the "default type" section of https://blog.golang.org/constants

vitesse-ftian

comment created time in a month

issue commentgolang/go

runtime: use clock_gettime instead of gettimeofday on Darwin

Go 1.15 will require 10.12. Setting that milestone.

randall77

comment created time in a month

issue commentgolang/go

cmd/compile: amd64 addressing modes are unwieldy

I hacked together another possible strategy to fix this. Take a look and let me know what you think.

josharian

comment created time in a month

issue commentgolang/go

cmd/compile: possible latent codegen issue on amd64 removing zero extensions

I've never been super happy about our whole sign extension story. In machine-independent space I think we're good, but we play very fast and loose in the machine-dependent space. Maybe there's some invariant we can use where we guarantee to only use the bits of a value that exist in its type. So there would be two effective ANDLconsts, one with a 32-bit type and one with a 64-bit type. ANDLconst [-1] could be removed in the former but not in the latter.

josharian

comment created time in a month

issue commentgolang/go

proposal: byte array <-> string conversions

Most systems nowadays are 64-bit. A string takes 16 bytes and a slice takes 24 bytes. It is especially useful if you want to:

stay compact with a small buffer

I don't see how this matters. Your first proposal would happen at compile time, and the second one just needs an extra 8 bytes of stack space (which could be optimized away if we cared).

provide human readable initial values for your byte buffer.

That seems like a reasonable thing to want. You could do

var ba [5]byte
copy(ba[:], "hello")

The only thing that [...]byte{"hello"} really gets you is automatic sizing.

jfcg

comment created time in a month

issue commentgolang/go

proposal: byte array <-> string conversions

The latter you can do with string(ba[:]). The former would only make sense with constant strings. Can you elaborate on situations where that comes up?

jfcg

comment created time in a month

issue commentgolang/go

proposal: cmd/compile: make 64-bit fields be 64-bit aligned on 32-bit systems, add //go:packed directive on structs

The same problem occurs for struct types.

type T struct { //go:align(8)
	a, b int32
}
func upconvert(p *struct{ a, b int32 }) *T {
	return (*T)(p)
}
danscales

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

I got in contact with the Corellium folks and they've opened an internal issue for it. Likely it will be fixed in their next release, eta unknown. I'll remove the 1.14 milestone. There's no more action required by us on this, other than to verify the fix when it is released.

bcmills

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

I can reproduce with C.

#include <stdio.h>

int geq(double x, double y) {
  return x >= y;
}

double x = 3;
double y = 4;

int main(int argc, char *argv[]) {
  for (int i = 0; i < 100000000; i++) {
    if (geq(x, y)) {
          printf("failed at iteration %d\n", i);
        }
    }
}
failed at iteration 532580
failed at iteration 19932913
failed at iteration 29020612
failed at iteration 32054460
failed at iteration 67123803
failed at iteration 70001529
failed at iteration 87241841
failed at iteration 89238362
failed at iteration 94347886

Note that it fails when compiled with arm-linux-gnueabihf-gcc (hard float) and not with arm-linux-gnueabi-gcc (soft float).

Anyone know how to contact Corellium and file a bug? Their website is not very helpful.

bcmills

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

Does it work if we disable async preemption? GODEBUG=asyncpreemptoff=1

Still buggy with that set. Same with GOGC=off.

arm64 binaries on the same machine work fine.

bcmills

comment created time in a month

issue commentgolang/go

proposal: cmd/compile: make 64-bit fields be 64-bit aligned on 32-bit systems, add //go:packed directive on structs

@ianlancetaylor Here's an example:

package main

type T int32 //go:align(8)

func upconvert(p *int32) *T {
	return (*T)(p)
}

If a 4-byte, but not 8-byte, aligned pointer were passed to upconvert, then the result would not obey the alignment constraints of T. If say, the *T was passed to assembly, it might assume 8-byte alignment, use an 8-byte aligned load instruction, and fail.

@beoran I'm not sure what you are asking. An interface is just a pair of pointers. They are aligned just as pointers are aligned (32 bits on 32 bit systems). In any case, the interface's alignment is irrelevant to the alignment of the thing it points to.

danscales

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

Same behavior for the compare-with-zero instruction, and for float64.

bcmills

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

Seems to be all the comparisons, occasionally. The error is always consistent. For ==, it's always a false negative. For all the other comparisons it is always a false positive.

package main

import "fmt"

//go:noinline
func geq(x, y float32) bool {
	return x >= y
}

//go:noinline
func leq(x, y float32) bool {
	return x <= y
}

//go:noinline
func gt(x, y float32) bool {
	return x > y
}

//go:noinline
func lt(x, y float32) bool {
	return x < y
}

//go:noinline
func eq(x, y float32) bool {
	return x == y
}

//go:noinline
func ne(x, y float32) bool {
	return x != y
}

func main() {
	for _, t := range []struct {
		name   string
		f      func(float32, float32) bool
		tx, ty float32 // inputs that should result in true
		fx, fy float32 // inputs that should result in false
	}{
		{">=", geq, 4, 3, 3, 4},
		{"<=", leq, 3, 4, 4, 3},
		{">", gt, 4, 3, 3, 4},
		{"<", lt, 3, 4, 4, 3},
		{"==", eq, 3, 3, 3, 4},
		{"!=", ne, 3, 4, 3, 3},
	} {
		for i := 0; i < 2e7; i++ {
			if !t.f(t.tx, t.ty) {
				fmt.Printf("bad %f %s %f, want true: %d\n", t.tx, t.name, t.ty, i)
			}
			if t.f(t.fx, t.fy) {
				fmt.Printf("bad %f %s %f, want false: %d\n", t.fx, t.name, t.fy, i)
			}
		}
	}
}
bad 3.000000 >= 4.000000, want false: 25635
bad 3.000000 >= 4.000000, want false: 69076
bad 3.000000 >= 4.000000, want false: 4916526
bad 3.000000 >= 4.000000, want false: 9937096
bad 3.000000 >= 4.000000, want false: 12789690
bad 3.000000 >= 4.000000, want false: 19958632
bad 4.000000 <= 3.000000, want false: 462589
bad 4.000000 <= 3.000000, want false: 14222185
bad 4.000000 <= 3.000000, want false: 15495103
bad 3.000000 > 4.000000, want false: 5855504
bad 3.000000 > 4.000000, want false: 7465563
bad 3.000000 > 4.000000, want false: 14899503
bad 3.000000 > 4.000000, want false: 16291141
bad 3.000000 > 4.000000, want false: 16402125
bad 3.000000 > 4.000000, want false: 17907133
bad 4.000000 < 3.000000, want false: 9664194
bad 4.000000 < 3.000000, want false: 13493741
bad 4.000000 < 3.000000, want false: 16757572
bad 3.000000 == 3.000000, want true: 2776863
bad 3.000000 == 3.000000, want true: 6072085
bad 3.000000 == 3.000000, want true: 10121837
bad 3.000000 == 3.000000, want true: 12263554
bad 3.000000 == 3.000000, want true: 15621010
bad 3.000000 == 3.000000, want true: 18710498
bad 3.000000 != 3.000000, want false: 1481130
bad 3.000000 != 3.000000, want false: 1484425
bad 3.000000 != 3.000000, want false: 1487792
bad 3.000000 != 3.000000, want false: 3018792
bad 3.000000 != 3.000000, want false: 19224977
bcmills

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

... or maybe the OS isn't saving/restoring floating point comparison flags? That would explain the erraticness...

bcmills

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

Here's a simple reproducer:

package main

import "fmt"

//go:noinline
func geq(x, y float32) bool {
	return x >= y
}

func main() {
	var x float32 = 3.0
	var y float32 = 4.0
	for i := 0; i < 1e8; i++ {
		if geq(x, y) {
			fmt.Printf("bad at iteration %d\n", i)
		}
	}
}

Compile this with GOARCH=arm GOARM=7, copy to builder, and run it. We get:

bad at iteration 189792
bad at iteration 251153
bad at iteration 3142582
bad at iteration 8808617
bad at iteration 12425345
bad at iteration 16630084
bad at iteration 16636778
bad at iteration 22789256
bad at iteration 29942954
bad at iteration 32523884
bad at iteration 32766014
bad at iteration 50490757
bad at iteration 60613276
bad at iteration 61778799
bad at iteration 71722383
bad at iteration 72913204
bad at iteration 76562474
bad at iteration 76569282
bad at iteration 79995821

Something is seriously wrong with the floating point comparison hardware on this machine.

The code for geq is just:

TEXT main.geq(SB) /usr/local/google/home/khr/gowork/tmp1.go
  tmp1.go:7             0x9d620                 ed9d0a01                MOVF 0x4(R13), F0       
  tmp1.go:7             0x9d624                 ed9d1a02                MOVF 0x8(R13), F1       
  tmp1.go:7             0x9d628                 eeb40ac1                CMPF F1, F0             
  tmp1.go:7             0x9d62c                 eef1fa10                MOVW FPSCR, APSR_NZCV   
  tmp1.go:7             0x9d630                 e3a00000                MOVW $0, R0             
  tmp1.go:7             0x9d634                 a3a00001                MOVW.GE $1, R0          
  tmp1.go:7             0x9d638                 e5cd000c                MOVB R0, 0xc(R13)       
  tmp1.go:7             0x9d63c                 e28ef000                ADD $0, R14, R15        
bcmills

comment created time in a month

issue commentgolang/go

cmd/compile: floating-point broken on android-arm-corellium builder

Seems to be failing for GOARCH=arm GOARM=6. GOARM=7 is also broken. GOARM=5 is fine.

$ GOARCH=arm GOARM=6 ../bin/go test -c math/rand
$ gomote push user-khr-android-arm-corellium-0
$ gomote run user-khr-android-arm-corellium-0 go/src/rand.test -test.run=Float32 -test.short -test.v
bcmills

comment created time in a month

issue commentgolang/go

math/rand: floating-point broken on android-arm-corellium builder

@benshi001 Ben, can you take a look? As far as I can tell floating-point compares are broken. The values printed in the error messages look correct.

bcmills

comment created time in a month

issue closedgolang/go

cache

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

<pre> $ go version

</pre>

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

<details><summary><code>go env</code> Output</summary><br><pre> $ go env

</pre></details>

What did you do?

<!-- If possible, provide a recipe for reproducing the error. A complete runnable program is good. A link on play.golang.org is best. -->

What did you expect to see?

What did you see instead?

closed time in a month

vishalkuo

issue commentgolang/go

cache

Empty, closing.

vishalkuo

comment created time in a month

issue commentgolang/go

runtime/race: race detector sometimes misses race on []byte copy

I think this is a "bug" (for some definition of bug of course b/c it was totally not obvious). I think we need to swap the order. If ranges don't overlap, it should not matter (minus potential effect on pseudo-random eviction). But if they overlap, then writes are always "stronger" for race detection and it's better to preserve them. This may be enough to "fix" this particular case. Currently tsan does remember reads to the region, but since the second access is also a read (slicebytetostring), reads don't race with reads, and the race is not detected. If we keep writes, it should be detected.

Yes, I noticed this as well, and swapping the order fixes this particular issue. I didn't understand why; now I do. I'll submit a CL for that for 1.15.

rogpeppe

comment created time in a month

issue commentgolang/go

reflect: map iteration does unnecessary excessive allocation for non-pointer types

Zero is being handled in #33136 .

I agree that the channel ones are probably not worth worrying about, performance-wise.

Convert I'm not sure.

But Slice (and the ones that call Slice, like Append and AppendSlice) would be even more important than the map ones, I suspect. @ugorji I didn't understand how you think Append could be made allocation free - could you explain?

ugorji

comment created time in a month

issue commentgolang/go

runtime/race: race detector sometimes misses race on []byte copy

Definitely strange. It seems very fragile. Changing the string length affects it (only lengths 10 and 11 demonstrate the bug). Replacing time.Sleep with a simple integer global variable write still reproduces the bug. Anything that calls into the race detector, really.

The calls out to the race detector from the runtime look fine, and the same in both cases.

slicecopy write 0xc00011c090 9
slicecopy read 0xc00011c092 9
slicebytetostring read 0xc00011c090 11

I suspect something in the race detector. Possibly this is just a limitation of the race detector, something is colliding in the shadow space in one case and not in another. It's not guaranteed to find all races.

rogpeppe

comment created time in a month

issue commentgolang/go

runtime: non-blocking read on an empty closed channel takes the slow path

Reopened because fixing CL has been reverted.

bemasc

comment created time in a month

IssuesEvent

issue commentgolang/go

CGO Unsetenv does not work on macOS 10.15.2

Sure, that's fine.

davidzech

comment created time in a month

issue commentgolang/go

CGO Unsetenv does not work on macOS 10.15.2

Simultaneous CLs, oops. Your CL needs a test. Feel free to grab the test from my CL and add it to your CL if you want. Or just review my CL.

davidzech

comment created time in a month

issue commentgolang/go

CGO Unsetenv does not work on macOS 10.15.2

Yep, this is indeed a bug. I don't think this ever worked. It's been broken since at least 1.11 on both darwin and linux. @davidzech You're right, x_cgo_unsetenv is wrong. Fix coming.

davidzech

comment created time in a month

issue closedgolang/go

sync/atomic: add `//go:noescape` to atomic functions accepting pointer

What version of Go are you using (go version)?

<pre> $ go version go version devel +cfe3cd903f Tue Jan 14 19:54:20 2020 +0000 linux/amd64 </pre>

Does this issue reproduce with the latest release?

yes, it is reproducible in go1.13 and go tip.

What did you do?

Run the following benchmark:

func BenchmarkAtomicAddUint64(b *testing.B) {
        b.ReportAllocs()
        for i := 0; i < b.N; i++ {
                n := uint64(123)
                sink += atomic.AddUint64(&n, 1)
        }
}

var sink uint64

What did you expect to see?

Zero memory allocations in the benchmark loop.

What did you see instead?

A memory allocation. The following is the output from go tool pprof --alloc_objects:

  25919883   25919883 (flat, cum)   100% of Total
         .          .     10:)
         .          .     11:
         .          .     12:func BenchmarkAtomicAddUint64(b *testing.B) {
         .          .     13:	b.ReportAllocs()
         .          .     14:	for i := 0; i < b.N; i++ {
  25919883   25919883     15:		n := uint64(123)
         .          .     16:		sink += atomic.AddUint64(&n, 1)
         .          .     17:	}
         .          .     18:}
         .          .     19:
         .          .     20:var sink uint64

It looks like Go compiler thinks that args passed to sync/atomic functions may escape.

Possible solution is to add //go:noescape to these functions.

closed time in a month

valyala

issue commentgolang/go

sync/atomic: add `//go:noescape` to atomic functions accepting pointer

Closing as duplicate of #16241

valyala

comment created time in a month

issue commentgolang/go

Inaccurate string to float64 conversion in strconv.ParseFloat

I can confirm. It seems like the fast path in strconv/atof.go:627, particularly the AssignDecimal call, is to blame. If I turn the optimize flag off, the bug goes away. This has been an issue since at least 1.11. @griesemer

g7r

comment created time in a month

issue commentgolang/go

panic: runtime error: slice bounds out of range

I think he's working with 1.12 or earlier, because the panic message has no array bounds contents, and that change went in for 1.13. I also matched up the trace entry in http/server/server.go:1995 and it makes sense in 1.12 but not 1.13 or tip.

The bogus line number was also introduced in 1.13, so this may be related, but is a different error than #36570 .

Could the original poster try this with 1.13, 1.14beta1, or tip? It is entirely possible it is fixed.

Also what @ianlancetaylor said - please fill out the template so we don't have to guess as to what Go version / OS / etc. you're using.

javasgl

comment created time in a month

issue commentgolang/go

cmd/internal/obj/x86: pad jumps to avoid Intel erratum

The additional benchmark I'd like to see is before and after the Go padding CL, on a chip that doesn't have the underlying bug. On an AMD chip, for example.

Some more info:

From here:

Below is the LLVM test suite we measured including the performance, code size and build time. The data indicates some performance effect (1.7%) from the microcode update, which was reduced to 0.5% with the SW mitigation of prefix padding. The code size increase in test suite is ~0.5%. And the compile time increase is ~2%. Comparing with hw_sw_prefix and hw_sw_nop, the exec_time difference is within -0.5%~0.5%, which may be a within the margin of error. Comparing with hw_sw_prefix and hw_sw_prefix_align_all, the exec_time difference is even less at 0.1%. Given that LLVM test-suite is a relatively small benchmark, we do not conclude which padding is preferable, hw_sw_prefix, hw_sw_nop or hw_sw_prefix_align_all.

The padding buys us back 1.2% in performance for a cost of 0.5% code size. There's only weak evidence that prefixes are better than nops.

From here:

Comparing with the hw_sw_prefix (prefix padding) with hw_sw_nop (nop padding) of SW mitigation, the hw_sw_prefix can provide better performance (0.3%~0.5% in geomean). In individual cases, we have observed a 1.4% performance improvement in prefix padding vs. nop padding. Comparing with sw_prefix with sw_nop on a system w/o MCU, we observed 0.7% better performance in sw_prefix. We also measured the increase in code size due to the padding to instructions to align branches correctly (Table 4). The geomean code size increase is 2-3% in both prefix padding and nop padding, with the individual outliers up to 4%.

This set of benchmarks shows prefixes are better than nops by 0.3-0.5%. They are paying 2-3% in code size.

In all of the LLVM discussion, I didn't see any mention of performance cost on non-affected chips. Maybe that's something they don't have to deal with, but we do.

The Go benchmarks posted by @markdryan show a performance improvement of 3.5% for a space cost of 1.4-2%. That seems quite a bit better than the C benchmarks were demonstrating. Not sure why that would be. (Maybe Go code is more branchy?)

My opinion is we should just do nothing. This is tricky (see all the discussion in here about how this breaks debuggers in various ways), and the performance deltas just aren't that large.

rsc

comment created time in a month

issue commentgolang/go

cmd/compile: doc: no description of option `-wb=false`

I don't think we want to document this flag. It's not intended for the end user. Its mere existence is an accident of history, when we were moving from a non-write-barrier GC to a write-barrier GC we needed a way to turn the write barrier on/off for testing. Maybe we should just remove it. Go just won't work with the write barrier off. Unless you also turn the GC off (GOGC=off). But that really isn't a tenable way to run Go either.

xaionaro

comment created time in a month

issue commentgolang/go

cmd/go: dylib on macos with rpath fails to run with go run/test

I'm not worried about hardening the build tool - that's just something that is being forced on us by Apple.

But this issue demonstrates that it just isn't reliable to depend on the go tool faithfully transmitting all the environment variables. Through no fault of our own. What other environment variables is Apple silently squashing? What new ones are they going to start squashing tomorrow?

I suspect setting the first entitlement described above would fix this particular issue. And maybe we should do that, for backwards compatibility if nothing else. But I don't think the underlying problem is solved by doing that. The underlying problem is only solved by build/run separately, or using the -exec flag as I mentioned.

edaniels

comment created time in a month

issue commentgolang/go

cmd/go: dylib on macos with rpath fails to run with go run/test

That first one might possibly fix this issue. I don't think we would want the second one. We don't want to actually allow unvalidated libraries to be run by the go tool. This entitlement is one that Go-generated binaries would need (assuming they were hardened, which they aren't at the moment) to use LD_LIBRARY_PATH successfully.

I don't think that's really the right way to go though. It's kind of an accident that the go tool passes environment variables unmolested through to temporary executables. It's asking for issues related to the same environment variable meaning different things in the go binary and the user's binary. (e.g. what if you wanted a different CC value for the go tool and the user's binary? Can't do that with CC=gcc go run.)

I think it is reasonable to ask that if you want to pass environment variables to a program written in Go, you have to build and run as separate steps.

You might be able to use the -exec flag to go run to introduce an environment variable at the right time. Something like

go run -exec "env LD_LIBRARY_PATH=/my/library/path" myprogram.go
edaniels

comment created time in a month

issue commentgolang/go

proposal: reflect: add Value.SetZero

My CL is simpler but has a performance discontinuity depending on size. Ian's is more invasive but will handle all sizes.

dsnet

comment created time in a month

issue commentgolang/go

^0 is interpreted as -1

Use x := ^uint64(0) instead.

kstenerud

comment created time in a month

issue commentgolang/go

cmd/go: dylib on macos with rpath fails to run with go run/test

That would make a lot of sense then.

From an article about the hardened runtime:

This involves a series of restrictions being placed on that app, which are intended to protect that app’s runtime integrity from certain types of exploits used by malware. These include code injection, the hijacking of dynamically-linked libraries (DLLs), and tampering with the app’s memory space.

So hard to be sure, but I suspect that part of hardening is squashing any LD_LIBRARY_PATH settings (the second part of that second sentence). So LD_LIBRARY_PATH won't propagate through the go binary to the binary to be run. Note that we don't generate hardened binaries. It is only the Go tool itself that is hardened. That's why it works if you build and run separately.

I think that although this is unfortunate, it's intentional and there's nothing we can do.

Just wait until Apple starts forcing us to generate hardened binaries...

edaniels

comment created time in a month

issue commentgolang/go

cmd/go: dylib on macos with rpath fails to run with go run/test

@cagedmantis Anything different about the release process for 1.13.6 from 1.13.5? None of the patches in 1.13.6 look relevant, and building 1.13.6 on my machine from source does not demonstrate the problem. Different temp directory? Different environment variables? Not sure what might matter...

edaniels

comment created time in a month

issue commentgolang/go

cmd/go: dylib on macos with rpath fails to run with go run/test

I take that back, I can reproduce. It only reproduces with the downloaded Go pkg, not the source.

edaniels

comment created time in a month

issue commentgolang/go

cmd/go: dylib on macos with rpath fails to run with go run/test

I can't reproduce this problem. I downloaded your test case, and edited the makefile to remove the GOROOT assignment (not needed) and change the instance of /usr/local/gogit/bin/go to go. When I run make with Go 1.13.6 on OSX 10.15.2, the makefile runs to completion with no errors.

A possible theory: your GOROOT, go, and/or /usr/local/gogit/bin/go point to more than one installation of Go.

edaniels

comment created time in a month

issue commentgolang/go

cmd/objdump: panic: runtime error: index out of range [1048574] when disassembling empty infinite loop

If I dump the binary without -S, I get this:

TEXT main.main(SB) /Users/khr/gowork/issue36570.go
  issue36570.go:4	0x1051570		90			NOPL			
  issue36570.go:1048575	0x1051571		ebfd			JMP main.main(SB)	

Looks like the line number is bad? I don't know offhand where the disassembler gets its line info from. This may be a case where we just protect the lookups against bad line numbers - it's definitely a weird edge case.

@dr2chase

bigwhite

comment created time in a month

issue commentgolang/go

runtime: Invalid garbage collection of structure pointers

So I guess this line:

dst := (*Dirent)(unsafe.Pointer(&b[0])) from https://golang.org/src/syscall/fs_nacl.go which casts a Dirent (https://golang.org/pkg/syscall/#Dirent) over a byte array, is also incorrect?

Yes, that code violates the rule. I wouldn't call it a major violation, as both Dirent and the byte slice backing store have no pointers in them. But doing casts like this can have Go version and platform dependent behavior (especially in this case, layout and alignment issues). Fortunately in package syscall in the stdlib, we can control for both platform and Go version.

In other word is casting a structure over a byte array fine as long as there are no Go pointers/references inside of it? Because I see casting structs over byte arrays in several occasions (I can find other examples if you like).

It's not fine according to the rules. You might run into issues doing this. But you're right that the damage should be limited. I do not expect you would run into GC issues because no pointers are involved. If you're prepared to accept possible Go version and platform-specific behavior on operations involving just that structure, go ahead.

Really though, you should write encoders and decoders for your data structure to and from []byte. That will ensure you avoid this issue altogether. For instance, in fs_nacl.go we should really do:

import "encoding/binary"
binary.LittleEndian.PutUint64(b[0:8], uint64(src.inode.Ino))
binary.LittleEndian.PutUint64(b[8:16], offset)

and so on. It will even generate the exact same code on architectures that support unaligned writes.

I understand that Go does not guarantee this to continue to work in the future, but I just want to ensure that this GC issue we are witnessing is not a symptom of another issue which is being hidden by instead pointing the finger on how we cast structs over byte arrays.

If you are casting pointer-containing structures onto a byte array, then I can definitely see that causing GC issues. If you have a program that just casts between pointer-free types and still has garbage collection problems, I'd like to see it.

abarisani

comment created time in a month

issue commentgolang/go

cmd/compile: inconsistent float64 behaviour between arm64 and amd64

You can only prohibit the fusing with an explicit float64 cast.

kortschak

comment created time in a month

issue commentgolang/go

cmd/compile: inconsistent float64 behaviour between arm64 and amd64

Yes, this is the arm64 backend using a fused multiply-subtract to compute the result. invNeg is not exactly 1/10, so 1 - (~1/10)*10 is not exactly zero. But (~1/10)*10 is within an epsilon of 1 (epsilon is around 10^-16 here), so the intermediate rounding gives a result of 0 on other platforms.

kortschak

comment created time in a month

issue commentgolang/go

testing.AllocsPerRun reports unexpected allocs count when -race is enabled

In cmd/compile/internal/gc/walk.go:walkCheckPtrArithmetic, I think we're marking the wrong node as not escaping. We need to mark the backing store of the slice, not the slice itself. This hack fixes the problem:

diff --git a/src/cmd/compile/internal/gc/walk.go b/src/cmd/compile/internal/gc/walk.go
index ef88db4742..7cc94c7ff1 100644
--- a/src/cmd/compile/internal/gc/walk.go
+++ b/src/cmd/compile/internal/gc/walk.go
@@ -4067,7 +4067,9 @@ func walkCheckPtrArithmetic(n *Node, init *Nodes) *Node {
 
        n = cheapexpr(n, init)
 
-       slice := mkdotargslice(types.NewSlice(types.Types[TUNSAFEPTR]), originals, init, nil)
+       ddd := nod(OXXX, nil, nil)
+       ddd.Esc = EscNone
+       slice := mkdotargslice(types.NewSlice(types.Types[TUNSAFEPTR]), originals, init, ddd)
        slice.Esc = EscNone
        slice.SetTransient(true) 

Probably want something more explicit than that, though.

lni

comment created time in a month

issue commentgolang/go

testing.AllocsPerRun reports unexpected allocs count when -race is enabled

So it isn't the poolLocal{}. It's the checking of the pointer arithmetic. If I mark indexLocal as //go:noninline, then look at its assembly when built with -race, I see an allocation of something being passed to runtime.checkptrArithmetic.

TEXT _sync.indexLocal(SB) /Users/khr/sandbox/readonly/src/sync/pool.go
func indexLocal(l unsafe.Pointer, i int) *poolLocal {
  0x10b9e10             65488b0c2530000000      MOVQ GS:0x30, CX                
  0x10b9e19             483b6110                CMPQ 0x10(CX), SP               
  0x10b9e1d             0f86bf000000            JBE 0x10b9ee2                   
  0x10b9e23             4883ec30                SUBQ $0x30, SP                  
  0x10b9e27             48896c2428              MOVQ BP, 0x28(SP)               
  0x10b9e2c             488d6c2428              LEAQ 0x28(SP), BP               
  0x10b9e31             488b442430              MOVQ 0x30(SP), AX               
  0x10b9e36             48890424                MOVQ AX, 0(SP)                  
  0x10b9e3a             e85191feff              CALL _runtime.racefuncenter(SB) 
        lp := unsafe.Pointer(uintptr(l) + uintptr(i)*128)
  0x10b9e3f             488d05daef1100          LEAQ _runtime.rodata+91616(SB), AX      
  0x10b9e46             48890424                MOVQ AX, 0(SP)                          
  0x10b9e4a             e81108f9ff              CALL _runtime.newobject(SB)             
  0x10b9e4f             488b442440              MOVQ 0x40(SP), AX                       
  0x10b9e54             48c1e007                SHLQ $0x7, AX                           
  0x10b9e58             488b4c2438              MOVQ 0x38(SP), CX                       
  0x10b9e5d             4801c8                  ADDQ CX, AX                             
  0x10b9e60             488b7c2408              MOVQ 0x8(SP), DI                        
  0x10b9e65             833d14fc2c0000          CMPL $0x0, _runtime.writeBarrier(SB)    
  0x10b9e6c             7564                    JNE 0x10b9ed2                           
  0x10b9e6e             48890f                  MOVQ CX, 0(DI)                          
  0x10b9e71             4889442420              MOVQ AX, 0x20(SP)                       
  0x10b9e76             48890424                MOVQ AX, 0(SP)                          
  0x10b9e7a             48897c2408              MOVQ DI, 0x8(SP)                        
  0x10b9e7f             48c744241001000000      MOVQ $0x1, 0x10(SP)                     
  0x10b9e88             48c744241801000000      MOVQ $0x1, 0x18(SP)                     
  0x10b9e91             e8caa2f8ff              CALL _runtime.checkptrArithmetic(SB)    
        return (*poolLocal)(lp)
  0x10b9e96             488b442420              MOVQ 0x20(SP), AX                       
  0x10b9e9b             48890424                MOVQ AX, 0(SP)                          
  0x10b9e9f             488d0d7a291300          LEAQ _runtime.rodata+172000(SB), CX     
  0x10b9ea6             48894c2408              MOVQ CX, 0x8(SP)                        
  0x10b9eab             48c744241001000000      MOVQ $0x1, 0x10(SP)                     
  0x10b9eb4             e8c7a1f8ff              CALL _runtime.checkptrAlignment(SB)     
  0x10b9eb9             e81291feff              CALL _runtime.racefuncexit(SB)          
  0x10b9ebe             488b442420              MOVQ 0x20(SP), AX                       
  0x10b9ec3             4889442448              MOVQ AX, 0x48(SP)                       
  0x10b9ec8             488b6c2428              MOVQ 0x28(SP), BP                       
  0x10b9ecd             4883c430                ADDQ $0x30, SP                          
  0x10b9ed1             c3                      RET                                     
        lp := unsafe.Pointer(uintptr(l) + uintptr(i)*128)
  0x10b9ed2             4889c2                  MOVQ AX, DX                             
  0x10b9ed5             4889c8                  MOVQ CX, AX                             
  0x10b9ed8             e85375feff              CALL _runtime.gcWriteBarrier(SB)        
  0x10b9edd             4889d0                  MOVQ DX, AX                             
  0x10b9ee0             eb8f                    JMP 0x10b9e71                           
func indexLocal(l unsafe.Pointer, i int) *poolLocal {
  0x10b9ee2             e8a955feff              CALL _runtime.morestack_noctxt(SB)      
  0x10b9ee7             e924ffffff              JMP _sync.indexLocal(SB)                

Somehow the compiler is deciding that the originals argument to checkptrArithmetic is escaping?

@mdempsky

lni

comment created time in a month

issue commentgolang/go

testing.AllocsPerRun reports unexpected allocs count when -race is enabled

Here's a somewhat simpler reproducer:

package alloctest

import (
  "sync"
  "testing"
)

func TestAlloc(t *testing.T) {
  p := &sync.Pool{}
  hit := 0
  miss := 0
  ac := testing.AllocsPerRun(1000000, func() {
    var v *int
    x := p.Get()
    if x != nil {
       hit++
       v = x.(*int)
    } else {
       miss++
       v = new(int)
    }
    p.Put(v)
  })
  t.Fatalf("allocs:%f hitRate:%f", ac, float64(hit)/float64(hit+miss))
}

Additionally, apply this hack to package testing to prevent AllocsPerRun from rounding to integer:

--- a/src/testing/allocs.go
+++ b/src/testing/allocs.go
@@ -41,5 +41,5 @@ func AllocsPerRun(runs int, f func()) (avg float64) {
        // We are forced to return a float64 because the API is silly, but do
        // the division as integers so we can ask if AllocsPerRun()==1
        // instead of AllocsPerRun()<2.
-       return float64(mallocs / uint64(runs))
+       return float64(mallocs) / float64(runs)
 }

WIthout -race, both go 1.13 and tip produce:

    issue36516_test.go:24: allocs:0.000000 hitRate:0.999999

With -race, go 1.13 produces:

issue36516_test.go:24: allocs:0.250180 hitRate:0.749819

So the race detector makes sync.Pool less effective. Not really sure what's going on there, but it isn't this issue. This happens for both 1.13 and tip.

With -race, tip produces:

    issue36516_test.go:24: allocs:2.249870 hitRate:0.750143

So we're getting an extra 2 allocations per iteration. The heap profiler says they are coming from here:

(pprof) top10
Showing nodes accounting for 162MB, 100% of 162MB total
      flat  flat%   sum%        cum   cum%
  147.50MB 91.05% 91.05%   147.50MB 91.05%  sync.indexLocal (inline)
   14.50MB  8.95%   100%      162MB   100%  _/Users/khr/gowork/issue36516.TestAlloc.func1
         .          .    276:func indexLocal(l unsafe.Pointer, i int) *poolLocal {
  147.50MB   147.50MB    277:	lp := unsafe.Pointer(uintptr(l) + uintptr(i)*unsafe.Sizeof(poolLocal{}))
         .          .    278:	return (*poolLocal)(lp)
         .          .    279:}

For some reason, with -race on the poolLocal{} is allocating. Weird.

lni

comment created time in a month

issue commentgolang/go

cmd/compile: depth limit reached in type formatting routine

The security hole aspects of this issue have been fixed. Removing release blocker and punting to 1.15.

griesemer

comment created time in 2 months

issue commentgolang/go

cmd/compile: constant propagation in compiler converts signaling NaN to quiet NaN

Never mind, addressability is not required, as we can make an addressable copy. You can do this:

package main

import (
	"fmt"
	"math"
	"reflect"
)

type myFloat32 float32

var qNaN32 = myFloat32(math.Float32frombits(0x7fc00000))
var sNaN32 = myFloat32(math.Float32frombits(0x7f800001))

func main() {
	v := reflect.ValueOf(sNaN32)
	w := reflect.ValueOf(qNaN32)

	fmt.Printf("%x\n", bits(v))
	fmt.Printf("%x\n", bits(w))
}

func bits(v reflect.Value) uint32 {
	p := reflect.New(v.Type())
	p.Elem().Set(v)
	return math.Float32bits(p.Convert(reflect.TypeOf((*float32)(nil))).Elem().Interface().(float32))
}
fxamacker

comment created time in 2 months

issue commentgolang/go

cmd/compile: constant propagation in compiler converts signaling NaN to quiet NaN

I asked around, and I don't think anyone is interested in adopting the reflect change for 1.14. It's just too late in the cycle. The release candidate will hopefully go out this week.

I did think of a possible workaround - if you can get an addressable reflect.Value containing the float32, you can get its value:

package main

import (
	"fmt"
	"math"
	"reflect"
)

type myFloat32 float32

var qNaN32 = myFloat32(math.Float32frombits(0x7fc00000))
var sNaN32 = myFloat32(math.Float32frombits(0x7f800001))

func main() {
	v := reflect.ValueOf(&sNaN32).Elem()
	w := reflect.ValueOf(&qNaN32).Elem()

	fmt.Printf("%x\n", bits(v))
	fmt.Printf("%x\n", bits(w))
}

// v must be an addressable value with underlying type float32.
func bits(v reflect.Value) uint32 {
	return math.Float32bits(v.Addr().Convert(reflect.TypeOf((*float32)(nil))).Elem().Interface().(float32))
}
fxamacker

comment created time in 2 months

issue closedgolang/go

runtime/race: use runtime.CallersFrames

The backtraces from the race detector don't include inlined frames. But we now have that information available, via runtime.CallersFrames. It'd be nice to use it.

I went to look at adding this myself, but I'd need a few pointers--the race detector code is out of tree, and I can't find the place where the race detector traceback interacts with the Go runtime.

cc @dvyukov

closed time in 2 months

josharian

issue commentgolang/go

runtime/race: use runtime.CallersFrames

Phew, thanks. I'm going to close this issue then.

josharian

comment created time in 2 months

issue commentgolang/go

runtime/race: use runtime.CallersFrames

Are you sure you saw a missing frame using 1.14? I don't see any missing frames. I get the same trace you did in #36431 where you said you turned inlining off. I didn't turn inlining off.

josharian

comment created time in 2 months

issue commentgolang/go

runtime/race: use runtime.CallersFrames

Hmmm, I thought I fixed this in the fix for #33309.

josharian

comment created time in 2 months

issue commentgolang/go

cmd/compile: constant propagation in compiler converts signaling NaN to quiet NaN

I'll ask around and see what other people think. Don't get your hopes up. From https://github.com/golang/go/wiki/Go-Release-Cycle, about the freeze:

This part of the release cycle is focused on improving the quality of the release, by testing it and fixing bugs that are found. However, every fix must be evaluated to balance the benefit of a possible fix against the cost of now having not as well tested code (the fix) in the release. Early in the release cycle, the balance tends toward accepting a fix. Late in the release cycle, the balance tends toward rejecting a fix, unless a case can be made that the fix is both low risk and high reward.

The fix is pretty low risk. But I don't think it qualifies as high reward. Of course, reasonable people can disagree about that...

fxamacker

comment created time in 2 months

more