profile
viewpoint

Ask questionsproposal: cmd/compile: make 64-bit fields be 64-bit aligned on 32-bit systems, add //go:packed directive on structs

Summary: We propose to change the default layout of structs on 32-bit systems such that 64-bit fields will be 64-bit aligned. The layout of structs on 64-bit systems will not change. For compatibility reasons (and finer control of struct layout), we also propose the addition of a //go:packed directive that applies to struct types. When the //go:packed directive is specified immediately before a struct type, then that struct will have a fully-packed layout, where fields are placed in order with no padding between them and hence no alignment based on types. The developer must explicitly add padding to enforce any desired alignment.

The main goal of this change is to avoid the bugs that frequently happen on 32-bit systems, where a developer wants to be able to do 64-bit operations on a 64-bit field (such as an atomic operation), but gets an alignment error because the field is not 64-bit aligned. With the current struct layout rules, a developer must often add explicit padding in order to make sure that such a 64-bit field is on a 64-bit boundary. As shown by repeated mentions in issue #599 (18 in 2019 alone), developers still often run into this problem. They may only run into it late in the development cycle as they are testing on 32-bit architectures, or when they execute an uncommon code path that requires the alignment.

As an example, the struct for ticks in runtime/runtime.go is declared as

var ticks struct {
	lock mutex
	pad  uint32 // ensure 8-byte alignment of val on 386
	val  uint64
}

so that the val field is properly aligned on 32-bit architectures. With the change in this proposal, it could be declared as:

var ticks struct {
	lock mutex
	val  uint64
}

and the val field would always be 64-bit aligned, even if other fields are added to struct.

We do not propose changing the alignment of 64-bit types which are arguments or return values, since that would change the Go calling ABI on 32-bit architectures. For consistency, we also don't propose to change the alignment of 64-bit types which are local variables. However, since we are changing the layout of some structs, we could be changing the ABI for functions that take a struct as an argument or return value. However, if assembly code is accessing struct fields, it should be using the symbolic constants (giving the offset of each field in a struct) that are available in go_asm.h (which is automatically generated for any package which contains an assembly file).

However, we do require a new directive for compatibility in the case of cgo and also interactions with the operating system. We propose the addition of a //go:packed directive that applies to struct types. When the //go:packed directive is specified immediately before a struct type, then that struct will have a fully-packed layout, where fields are placed in order with no padding between them and hence no alignment based on types. The developer must explicitly add padding fields to enforce any desired alignment within the structure.

We would then use the //go:packed directive for cgo-generated struct types (with explicit padding added as needed). Also, given the strict layout requirements for structures passed to system calls, we would use //go:packed for structs that are used for system calls (notably Flock_t and Stat_t, used by Linux system calls). We can enforce the use of //go:packed for structures that are passed to system calls (typically via pointers) with the use of a go vet check. I don't think we would particularly encourage the use of //go:packed in any other situations (as with most directives), but it might be useful in a few specific other cases.

This proposal is essentially a solution to issue #599. A related issue #19057 proposes a way to force the overall alignment of structures. That proposal does not propose changing the default alignment of 64-bit fields on 32-bit architectures (the source of the problems mentioned in issue #599), but would provide a mechanism to explicitly align 64-bit fields without using developer-computed padding. With that proposal, aligning a uint64 field (for example) in a struct to a 64-bit boundary on a 32-bit system would require replacing the uint64 type with a struct type that has the required 64-bit alignment (as I understand it).

Compatibility: We are not changing the alignment of arguments, return variables, or local variables. Since we would be changing the default layout of structs, we could affect some programs running on 32-bit systems that depend on the layout of structs. However, the layout of structs is not explicitly defined in the Go language spec, except for than minimum alignment, and we are maintaining the previous minimum alignments. So, I don't believe this change breaks the Go 1 compatibility promise. If assembly code is accessing struct fields, it should be using the symbolic constants (giving the offset of each field in a struct) that are available in go_asm.h. go_asm.h is automatically generated and available for each package that contains an assembler file, or can be explicitly generated for use elsewhere via go tool compile -asmhdr go_asm.h ....

golang/go

Answer questions dr2chase

@ianlancetaylor Or declaring that such alignment upconversions are unsafe -- and perhaps they are checked when they occur there (i.e., semantic change to a pointer cast that adds checking, but this can only occur in unsafe code).

My Great Idea may be a Great Idea™. (As a wise former colleague once remarked, the ™ means it isn't what it says it is. Would you rather have cheese, or Cheese™?)

useful!

Related questions

cmd/link: segmentation fault during mach-o linking hot 4
cmd/go: cannot find module providing package error stops `go get` processing hot 2
cmd/go: needs a better error than "missing dot in first path element" when GOROOT is set incorrectly hot 2
x/xerrors: fails to compile on tip hot 1
vendor/golang.org/x/xerrors/adaptor_go1_13.go:16:14: undefined: errors.Frame ... hot 1
cmd/go: `go clean <package>` downloads modules hot 1
cmd/cgo error: runtime: unknown pc 0x7fff5c805b86 hot 1
runtime: crash with "invalid pc-encoded table" hot 1
cmd/vet: potential false positive in the "suspect or" check hot 1
cmd/link: showing many ld warnings of "building for macOS, but linking in object file" hot 1
runtime: go program crach, it seems fall into infinite loop hot 1
cmd/go: major version without preceding tag must be v0, not v1 - breaks build of github.com/go-check hot 1
runtime: macOS Sierra builders spinning hot 1
cmd/go: Problem using go modules hot 1
cmd/go: "unrecognized import path" for local packages after updating to go1.13 hot 1
Github User Rank List