profile
viewpoint
Clément Chigot Helflym ATOS Grenoble, France

colljori/QemuSTM32 0

Modélisation SystemC-TLM d’IP et intégration système matériel/logiciel

fpetrot/nucleo 0

Nucleo platform model

fpetrot/rabbits-devel-env 0

Rabbits clone for Nucleo modeling

Helflym/afero 0

A FileSystem Abstraction System for Go

Helflym/beats 0

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash

Helflym/color 0

Color package for Go (golang)

Helflym/dhcp 0

DHCPv6 and DHCPv4 packet library, client and server written in Go

Helflym/flock 0

Thread-safe file locking library in Go (originally github.com/theckman/go-flock)

PR opened fsnotify/fsnotify

Add polling fallback

Polling fallback can be used by operating systems without any internal mechanism to watch files. This is only use on AIX for now, but should work on every over OS (tested on Linux).

+446 -12

0 comment

2 changed files

pr created time in 3 days

push eventHelflym/fsnotify

Clément Chigot

commit sha 8cddba14a52318b83de662376a5be585d55c4a25

Add polling fallback Polling fallback can be used by operating systems without any internal mechanism to watch files. This is only use on AIX for now, but should work on every over OS (tested on Linux).

view details

push time in 3 days

fork Helflym/fsnotify

Cross-platform file system notifications for Go.

https://fsnotify.org

fork in 3 days

delete branch Helflym/beats

delete branch : fsync

delete time in 4 days

pull request commentelastic/beats

libbeat/common/file: fix fsync failure on AIX

Thank you @Helflym. I did prepare the backports to the 7.6 release branch and 7.x development branch. See #16417 and #16418.

Nice ! However, note that it won't work until all dependencies update their code too and that we import their code in vendor afterwards. Except if you allow manual modifications of vendor which is what I've done for our RPMs.

Helflym

comment created time in 4 days

PR opened elastic/beats

libbeat/common/file: fix fsync failure on AIX

<!-- Type of change Please label this PR with one of the following labels, depending on the scope of your change:

  • Bug
  • Enhancement
  • Breaking change
  • Deprecation -->

What does this PR do?

<!-- Mandatory Explain here the changes you made on the PR. Please explain the WHAT: patterns used, algorithms implemented, design architecture, message processing, etc. --> On AIX, fsync syscall doesn't work with read-only files. As os.Open is actually opening in read-only mode, it must be changed for os.OpenFile with O_RDWR flag.

Why is it important?

<!-- Mandatory Explain here the WHY, or the rationale/motivation for the changes. --> Fix for AIX

Checklist

<!-- Mandatory Add a checklist of things that are required to be reviewed in order to have the PR approved

List here all the items you have verified BEFORE sending this PR. Please DO NOT remove any item, striking through those that do not apply. (Just in case, strikethrough uses two tildes. Scratch this.) -->

  • [x ] My code follows the style guidelines of this project
  • [ x] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works

How to test this PR locally

<!-- Recommended Explain here how this PR will be tested by the reviewer: commands, dependencies, steps, etc. --> Only for AIX

Related issues

<!-- Recommended Link related issues below. Insert the issue link or reference after the word "Closes" if merging this should automatically close it.

  • Closes #123
  • Relates #123
  • Requires #123
  • Superseds #123 -->
  • Related #15785
+46 -1

0 comment

2 changed files

pr created time in 6 days

create barnchHelflym/beats

branch : fsync

created branch time in 6 days

PR opened elastic/beats

Remove docker and kubernetes tests and metadata for AIX

<!-- Type of change Please label this PR with one of the following labels, depending on the scope of your change:

  • Bug
  • Enhancement
  • Breaking change
  • Deprecation -->

What does this PR do?

<!-- Mandatory Explain here the changes you made on the PR. Please explain the WHAT: patterns used, algorithms implemented, design architecture, message processing, etc. --> This PR disables tests based on docker or kubernetes features for AIX. It also forces docker and kubernetes to be disable by mage configuration files.

Why is it important?

<!-- Mandatory Explain here the WHY, or the rationale/motivation for the changes. --> Docker doesn't work on AIX thus tests or anything else linked to it shouldn't be made on AIX.

Checklist

<!-- Mandatory Add a checklist of things that are required to be reviewed in order to have the PR approved

List here all the items you have verified BEFORE sending this PR. Please DO NOT remove any item, striking through those that do not apply. (Just in case, strikethrough uses two tildes. Scratch this.) -->

  • [x] My code follows the style guidelines of this project
  • [x] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [x] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works

Author's Checklist

<!-- Recommended Add a checklist of things that are required to be reviewed in order to have the PR approved -->

  • [ ]

How to test this PR locally

<!-- Recommended Explain here how this PR will be tested by the reviewer: commands, dependencies, steps, etc. -->

This is only aimed to be tested on AIX, but the part disabling the tests can be tried on any OS by changing the uname condition.

Related issues

<!-- Recommended Link related issues below. Insert the issue link or reference after the word "Closes" if merging this should automatically close it.

  • Closes #123
  • Relates #123
  • Requires #123
  • Superseds #123 -->
  • Related #15785
+22 -3

0 comment

3 changed files

pr created time in 6 days

create barnchHelflym/beats

branch : remove-docker

created branch time in 6 days

fork Helflym/beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash

https://www.elastic.co/products/beats

fork in 9 days

issue commentelastic/beats

Add AIX support

A quick update to say that I've released a first beta version of filebeat, metricbeat and auditbeat under our website: http://www.bullfreeware.com/?searching=true&package=beat&from=&to=&libraries=false&exact=false&version=5. Any feedback about configuration and/or modules would be much appreciated.

I've started to push some patches for dependencies upstream but almost all of them are stuck right now. So, we'll have to wait to have AIX on the beats' upstream. https://github.com/elastic/gosigar/pull/133 https://github.com/elastic/go-sysinfo/pull/77 https://github.com/shirou/gopsutil/pull/834 https://github.com/fsnotify/fsnotify/issues/9

mostlyjason

comment created time in 9 days

PR opened shirou/gopsutil

net: add AIX support
+426 -1

0 comment

2 changed files

pr created time in 13 days

push eventHelflym/gopsutil

Clément Chigot

commit sha 77ff16c0819f07b7f331857e99c68540a63cdfa6

net: add AIX support

view details

push time in 13 days

PR closed shirou/gopsutil

net: move netstat parsing functions to net.go

As AIX is also using "netstat -an" in order to get connections, move netstat functions to net.go to make it available for both OpenBSD and AIX. Also rename lsof parsing functions to have Lsof in their name.

+90 -85

2 comments

3 changed files

Helflym

pr closed time in 13 days

pull request commentshirou/gopsutil

net: move netstat parsing functions to net.go

Some parts are similars, especially when dealing with net connections. But I understand your point of view, so I'll close this PR, and directly submit the AIX one with its own netstat parsing functions.

Helflym

comment created time in 13 days

issue closedgolang/go

os: SameFile should not only check inodes and devices

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

tip

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Every Unix systems (tested on AIX and Linux).

What did you do?

I'm calling os.SameFile to check if a file have been moved, using the two FileInfos associated with the old path and the new path. However, there is some case where the device and the inode are the same.

I don't have a pure Go example, as it isn't 100% reproducible. But, here is a bash version.

$ touch alpha && ls -i alpha
15863201 alpha
$ rm alpha
$ mkdir beta && ls -i
15863201 beta

As you can see, both files end up with the same inode. Therefore, os.SameFile on Unix systems will return true, even if it's one is a basic file and one a directory.

I think it's acceptable when two basic files are involved (it can be considered as a "move" and a "write" afterall), same for two directories. But for the shown case, it seems weird to me. I don't know if there is a way to transform a file to a directory directly.

Shouldn't os.SameFile also check IsDir() ?

closed time in 23 days

Helflym

issue commentgolang/go

os: SameFile should not only check inodes and devices

Indeed, I've thought about the fact that os.FileInfo isn't accurate once a file have been deleted. It still good to have this issue with anyone comes across a similar problem. I'll check if I can hash the file's content without impacting performances too much. Thanks @ianlancetaylor

Helflym

comment created time in 23 days

issue openedgolang/go

os: SameFile should not only check inodes and devices

<!-- Please answer these questions before submitting your issue. Thanks! For questions please use one of our forums: https://github.com/golang/go/wiki/Questions -->

What version of Go are you using (go version)?

tip

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Every Unix systems (tested on AIX and Linux).

What did you do?

I'm calling os.SameFile to check if a file have been moved, using the two FileInfos associated with the old path and the new path. However, there is some case where the device and the inode are the same.

I don't have a pure Go example, as it isn't 100% reproducible. But, here is a bash version.

$ touch alpha && ls -i alpha
15863201 alpha
$ rm alpha
$ mkdir beta && ls -i
15863201 beta

As you can see, both files end up with the same inode. Therefore, os.SameFile on Unix systems will return true, even if it's one is a basic file and one a directory.

I think it's acceptable when two basic files are involved (it can be considered as a "move" and a "write" afterall), same for two directories. But for the shown case, it seems weird to me. I don't know if there is a way to transform a file to a directory directly.

Shouldn't os.SameFile also check IsDir() ?

created time in 24 days

issue commentfsnotify/fsnotify

Polling fallback

I need this package in order to port auditbeat software (https://github.com/elastic/beats) on AIX. However, AIX doesn't provide an easier way to watch files than polling them. I'm planning to take some part of @radovskyb's work. But we don't have the end of your slack conversation here, @nathany and @radovskyb. I would like to know if any of you is against such thing ?

nathany

comment created time in 25 days

PR opened elastic/gosigar

Add AIX support

This Pull Request adds support for AIX Operating system which is now available on Go.

+574 -25

0 comment

8 changed files

pr created time in a month

push eventHelflym/gosigar

Clément Chigot

commit sha 4f394b700e5e6e64a5b7c3de1cc02170cac40e19

add AIX support Not all the features are available yet. Currently implemented: - Cpu - CpuList - FileSystemList - FileSystemUsage - LoadAverage - Mem - ProcArgs - ProcEnv - ProcExe - ProcList - ProcMem - ProcState - ProcTime - Rusage - Swap - Uptime

view details

Clément Chigot

commit sha 9749e3aa3c8374677d431dcc5ed57d35e788e3a3

test: trim environment variables given by ProcEnv Environment variable might have spaces at the end. Therefore, trim both entries before calling the assert function. Examples: ``` expected: "${debian_chroot:+($debian_chroot)}\\u@\\h:\\w\\$" actual: "${debian_chroot:+($debian_chroot)}\\u@\\h:\\w\\$ " ```

view details

push time in a month

issue commentgolang/go

x/review/git-codereview: skip tests that require "gofmt" binary when it's not available

gofmt is available on aix/ppc64. But for an unknown reason it was missing from the default path of the aix builder, likewise for go binary. I've added them so let see if it's alright now.

bcmills

comment created time in a month

push eventHelflym/go-sysinfo

Clément Chigot

commit sha c5c07ec7595a4a372dd83f818f43f1cd049a3b9a

all: add AIX provider Add a implementation without CGO for AIX. Host information are retrieved using AIX commands (uname, oslevel) Process information are retrieved by /proc folder. A CGO implementation could be added to more accurate syscalls and libraries (getprocs, libperfstat, etc). But github.com/elastic/gosigar provides such things. A few features are still missing: - Host CPU usage since boot time: there is no easy way to retrieve it. - Process' environment

view details

push time in a month

push eventHelflym/go-sysinfo

Clément Chigot

commit sha 4c0255b68afc326e6d1af55d75c3d44f54999637

all: add AIX provider Add a implementation without CGO for AIX. Host information are retrieved using AIX commands (uname, oslevel) Process information are retrieved by /proc folder. A CGO implementation could be added to more accurate syscalls and libraries (getprocs, libperfstat, etc). But github.com/elastic/gosigar provides such things. A few features are still missing: - Host CPU usage since boot time: there is no easy way to retrieve it. - Process' environment

view details

push time in a month

push eventHelflym/go-sysinfo

Clément Chigot

commit sha 3b3ab77ac3186b7898f8f5853c491c6a44172215

all: add AIX provider Add a implementation without CGO for AIX. Host information are retrieved using AIX commands (uname, oslevel) Process information are retrieved by /proc folder. A CGO implementation could be added to more accurate syscalls and libraries (getprocs, libperfstat, etc). But github.com/elastic/gosigar provides such things. A few features are still missing: - Host CPU usage since boot time: there is no easy way to retrieve it. - Process' environment

view details

push time in a month

PR opened elastic/go-sysinfo

all: add AIX provider

Add a implementation without CGO for AIX. Host information are retrieved using AIX commands (uname, oslevel) Process information are retrieved by /proc folder.

A CGO implementation could be added to more accurate syscalls and libraries (getprocs, libperfstat, etc). But github.com/elastic/gosigar provides such things.

A few features are still missing:

  • Host CPU usage since boot time: there is no easy way to retrieve it.
  • Process' environment
+888 -0

0 comment

11 changed files

pr created time in a month

push eventHelflym/go-sysinfo

Clément Chigot

commit sha 90e1c94486bdcc85164a370a2e5abfdafb0c657e

all: add AIX provider Add a implementation without CGO for AIX. Host information are retrieved using AIX commands (uname, oslevel) Process information are retrieved by /proc folder. A CGO implementation could be added to more accurate syscalls and libraries (getprocs, libperfstat, etc). But github.com/elastic/gosigar provides such things. A few features are still missing: - Host CPU usage since boot time: there is no easy way to retrieve it. - Process' environment

view details

push time in a month

fork Helflym/go-sysinfo

go-sysinfo is a library for collecting system information.

fork in a month

fork Helflym/gosigar

Gathers system and per process statistics

fork in a month

PR opened shirou/gopsutil

net: move netstat parsing functions to net.go

As AIX is also using "netstat -an" in order to get connections, move netstat functions to net.go to make it available for both OpenBSD and AIX. Also rename lsof parsing functions to have Lsof in their name.

+90 -85

0 comment

3 changed files

pr created time in a month

push eventHelflym/gopsutil

Clément Chigot

commit sha 3ac57409f98c87df716f255f46f3ddbcee6c0f1a

net: move netstat parsing functions to net.go As AIX is also using "netstat -an" in order to get connections, move netstat functions to net.go to make it available for both OpenBSD and AIX. Also rename lsof parsing functions to have Lsof in their name.

view details

push time in a month

fork Helflym/gopsutil

psutil for golang

fork in a month

issue commentelastic/beats

Add AIX support

I heard separately that metricbeat on AIX will require more effort due to the OS API being different. Are you working on that as well?

Yes of course. For the moment, it seems that porting metrics dependencies (gosigar, go-sysinfo, etc) is enough to have a working metricbeat. Maybe the data won't be fully accurate and some fields will be missing. But it will be enough for a first metricbeat, and I will have time to improve them if needed.

mostlyjason

comment created time in a month

issue commentelastic/beats

Add AIX support to Filebeat

Hi, I'm a member behind the Bullfreeware website and I'm currently actively porting Filebeat, Metricbeat and Auditbeat for AIX 7.2. Filebeat is already in good shape and I'll soon start pushing a few patches to introduce AIX to the beats software. Moreover, a rpm of version 7.5.1 should be able at the same time on our website, for those who are interested in.

About dependencies, as you said a fix has been merged to disable Docker and K8s for platforms which don't support it. A few adjustments have to be made in order to fully disable the build of Docker packages (especially with tests), but it's no longer a needed dependency for AIX. The other dependencies (for Filebeat, Metricbeat and Auditbeat) that I'm aware of are:

I'll keep posting updates on this issue. Note, maybe the title can be changed to "Add AIX support" as Filebeat isn't the only one targeted.

mostlyjason

comment created time in a month

issue commentelastic/beats

Add AIX support to Filebeat

Hi, I'm currently actively porting Filebeat, Metricbeat and Auditbeat for AIX 7.2. Filebeat is already in good shape and I'll soon push a few patches to introduce AIX to the beats environment.

About dependencies, as you said a fix has been merged to disable Docker and K8s for platforms which don't support it. A few adjustment has to be made in order to fully disable the build of Docker (especially with tests), but it's no longer a needed dependency for AIX. The other dependencies that I'm aware of are:

mostlyjason

comment created time in a month

pull request commentu-root/u-root

pkg/rand: add AIX support

If you're speaking about the whole "u-root", not at all (at least not right now). If you're speaking of this patch and "pkg/rand" package, it might be. At least, its compilation is needed by "github.com/insomniacslk/dhcp/dhcpv4" which is needed by Beats software (the ones that I'm porting right now). However, I don't know what these packages are planning to do with it.

Helflym

comment created time in a month

fork Helflym/godror

GO DRiver for ORacle DB

fork in a month

push eventHelflym/dhcp

Clément Chigot

commit sha 72e356462ba1d38e4bb7db61fdc4d0ed31c7e45a

interfaces: add support for AIX Signed-off-by: Clément Chigot <clement.chigot@atos.net>

view details

push time in a month

PR opened insomniacslk/dhcp

interfaces: add support for AIX
+1 -1

0 comment

1 changed file

pr created time in a month

push eventHelflym/dhcp

Clément Chigot

commit sha 9daecb3962dbeaa138084e9d1acf761d5ef081ef

interfaces: add support for AIX

view details

push time in a month

fork Helflym/dhcp

DHCPv6 and DHCPv4 packet library, client and server written in Go

fork in a month

push eventHelflym/u-root

Clément Chigot

commit sha 917ac00435a5d67e5fdb6605d436d50078fd2010

pkg/rand: add AIX support This package is sometimes used by other projects, at least "github.com/insomniacslk/dhcp/dhcpv4". Therefore, it must be ported even if the whole u-root project isn't available on AIX. Signed-off-by: Clément Chigot <clement.chigot@atos.net>

view details

push time in a month

PR opened u-root/u-root

pkg/rand: add AIX support

This package is sometimes used by other projects, at least "github.com/insomniacslk/dhcp/dhcpv4". Therefore, it must be ported even if the whole u-root project isn't available on AIX.

+2 -2

0 comment

2 changed files

pr created time in a month

push eventHelflym/u-root

Clément Chigot

commit sha 95d0b05896c89e8db201ec510d27f3eea2feecc1

pkg/rand: add AIX support This package is sometimes used by other projects, at least "github.com/insomniacslk/dhcp/dhcpv4". Therefore, it must be ported even if the whole u-root project isn't available on AIX.

view details

push time in a month

fork Helflym/u-root

A fully Go userland with Linux bootloaders! u-root can create a root file system (initramfs) containing a busybox-like set of tools written in Go.

http://u-root.tk

fork in a month

pull request commentgofrs/flock

Add AIX support with fcntl

Hi Theckman,

Any news about this PR ? gofrs/flock is used in many others projects like Beats software and at the moment, we have to manually update "vendor" folder if we want to build them. I know that you can't test any future modifications on an AIX environment by yourself but I suppose letting the AIX community fixing it if it ever goes wrong is a viable solution

Helflym

comment created time in 2 months

PR opened lib/pq

Add AIX support
+1 -1

0 comment

1 changed file

pr created time in 2 months

push eventHelflym/pq

Clément Chigot

commit sha 2a4ae662cd1ced5756f337963c738f82fafa853c

Add AIX support

view details

push time in 2 months

fork Helflym/pq

Pure Go Postgres driver for database/sql

http://godoc.org/github.com/lib/pq

fork in 2 months

issue commentgolang/go

runtime/pprof: failure in TestCPUProfileMultithreaded

AIX might be a bit slow when handling signals like SIGPROF there. But the test seems to end up with a 3.2s duration (as seen with pprof_test.go:285: retrying with 3.2s duration), so I'm not sure increasing the duration will change anything. Note that I've never seen this bug before.

bcmills

comment created time in 2 months

issue commentgolang/go

runtime: timeout with GDB tests on aix/ppc64

@mknyszek no, I'm sorry I thought it was related to the new page allocator. But indeed, as you said, it seems to be linked with 374c2847f9c03da7365bfb78e5ef96a0cb837656 according to a git bisect. Moreover, our GDB already have trouble handling signals, so it seems legit. I'm waiting for updates from the GDB team anyway.

Helflym

comment created time in 3 months

issue closedgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

Since the new page allocator (#35112) has been enabled by default, the runtime is extremely slow. Some recent failures on aix/ppc64 builder seems also related, starting from CL 190622. There is a timeout during runtime tests (cf https://build.golang.org/log/7e68765a5fe5e9887ef06fd90de1c9ae6682e73d https://build.golang.org/log/d09de5259a97061169ce2648541666ba1101fc1c)

Before CL 201765:

$ time ./make.bash 
Building Go cmd/dist using /opt/freeware/lib/golang. (go1.13.4 aix/ppc64)
Building Go toolchain1 using /opt/freeware/lib/golang.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for aix/ppc64.
---
Installed Go for aix/ppc64 in /opt/freeware/src/packages/BUILD/goroot
Installed commands in /opt/freeware/src/packages/BUILD/goroot/bin

real    1m52.246s
user    2m53.752s
sys     0m16.242s

After

$ time ./make.bash 
Building Go cmd/dist using /opt/freeware/lib/golang. (go1.13.4 aix/ppc64)
Building Go toolchain1 using /opt/freeware/lib/golang.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for aix/ppc64.
---
Installed Go for aix/ppc64 in /opt/freeware/src/packages/BUILD/goroot
Installed commands in /opt/freeware/src/packages/BUILD/goroot/bin

real    20m44.516s
user    4m56.656s
sys     23m5.166s

cc @mknyszek

closed time in 3 months

Helflym

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

@mknyszek yes I think we can close this issue for now.

Closed by https://github.com/golang/go/commit/5042317d6919d4c84557e04be35130430e8d1dd4

Helflym

comment created time in 3 months

issue openedgolang/go

runtime: timeout with GDB tests on aix/ppc64

https://build.golang.org/log/98c7edecebf8de8d333d58a2286729cd6d821cef https://build.golang.org/log/66fbb96c7aee8d5b6f191319ed3d8748f238255f https://build.golang.org/log/8fba4a8ce0f6fb1cd3170013f51ff2fdbc35d269

Our GDB on AIX is a bit unstable and it doesn't seem to like the new page allocator for an unknown reason. Even an helloworld can't be run properly. It might be related to #35451 but I think it's more a GDB problem. The team working on it has been contacted. So until now, I would like to disable these tests in order to release our builder.

created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

@Helflym Yeah that's what I figured. I think for this release we're going to stick with the 48-bit address space assumption as the fix.

I think that's better anyway. Even if it's not exactly how AIX is working, it will be closer to AIX and avoid unnecessary build failures everytime something dealing with the memory is changed.

If one day, there is other huge changes which might allow 60bits addresses again, just warn me and I'll try asap. But until then, let's keep AIX with an arena similar to Linux, it will be safer.

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

@mknyszek I've tried your patch with the 60bits addresses and that's far more quickier but not enough sadly.

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

Ok, thanks for your answers. And, if one day, the incremental mapping is implemented, we can still try to remove this arenaBaseOffset and see how AIX is performing. But at the moment, I'd rather have a working AIX builder and I don't think we have any other options, right ?

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

It would be better to use mprotect and still assume a 60-bit address space just in case AIX changes it's mmap policy (since this isn't documented anywhere).

What policy do you mean by this ? The fact that mmap addresses are starting after 0x0a00000000000000 ? I don't think it will happen in a near future and if it does anyway, there will be a way to still use this segment. AIX have a strict compatibility policy, everything compiled in a previous version must run as is in all the following one. Therefore, there is many ways to keep older behaviors when running an newly compiled process.

However, the fact that mmap takes 1 second to run makes this plan dead-on-arrival. Perhaps the arenaBaseOffset is the right way to go in this case, and to just deal with changes to AIX's mmap in the future?

mmap (and munmap afterwards) is taking so long because the memory area reserved is really huge. Isn't it possible to allocate s.chunks incrementally ? Or have several levels of chunks (like this is done in the arena with arenaL1 and arenaL2? At the moment, only AIX is facing issues but others OS might have the same problem in the future. Especially, because amd64 is already providing a way to mmap on 57 bits addresses (according to malloc.go)

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

Does it matter if any pages have been touched in the mapping?

It seems that slow munmap, yes.

I've been using this code.

#include <sys/mman.h>
#include <stdio.h>
#include <sys/time.h>

#define ALLOC_SIZE 0x200000000000ull
#define PAGE_ALLOC_NB 0x40000
#define GAP_BETWEEN_PAGES 0x100000ull


int main(void)
{
        void *addr;
	struct timeval start, stop;

        gettimeofday(&start, NULL);
        addr = mmap(NULL, ALLOC_SIZE, PROT_NONE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0);
	gettimeofday(&stop, NULL);
        printf("memory mapped at %p (%d µs)\n", addr, (stop.tv_sec - start.tv_sec) * 1000000 + stop.tv_usec - start.tv_usec);

	for (int i=0x0; i < PAGE_ALLOC_NB; i++){
		int * n_addr = ((int *)(addr + i*GAP_BETWEEN_PAGES));
		mprotect(n_addr, 0x1000, PROT_WRITE|PROT_READ);
		*n_addr = i;
		/* if (i % 0x1000) { */
		/* 	printf("%p = 0x%x\n", n_addr, *n_addr); */
		/* } */
	}
        fflush(stdout);

	gettimeofday(&start, NULL);
        munmap(addr, ALLOC_SIZE);
	gettimeofday(&stop, NULL);
        printf("memory unmap (%d µs)\n", (stop.tv_sec - start.tv_sec) * 1000000 + stop.tv_usec - start.tv_usec);
        fflush(stdout);

        return 0;
}

munmap performances are slowing down whenever ALLOC_SIZE, PAGE_ALLOC_NB or GAP_BETWEEN_PAGES is being increased. With the current settings, it takes ~3.6s. That's a lot but nothing compared to the one in Go which takes more than 30/40s. Note that I haven't any idea which PAGE_ALLOC_NB and GAP_BETWEEN_PAGES should be used to be closer to Go runtime.

Anyway, ALLOC_SIZE is the same than the Go one, and mmap takes more than 1s which is already too long if every Go programs (even an helloworld) takes more than 1s to start.

Note, using mprotect instead of munmap + mmap seems to work fine. I've tested it in Go runtime too. I'll check the performances.

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

I do agree with you. I've always wanted to use this arenaBaseOffset in order to avoid too much difference between aix/ppc64 and linux/ppc64 but I wasn't aware of (^0x0a00000000000000+1)&uintptrMask. Anyway, this is working fine. I'll create the patch tomorrow and check that nothing else has been broken.

Note that I've traced a little which unmapping is slowing the tests and it's indeed when releasing the object created by this sysReserve. However, I don't understand why this munmap is taking so long as it's far quicker when doing a similar mmap/munmap on C.

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

@Helflym OK, it would be nice to not have to support the full 60 bit address space if this is the cause of the performance slowdown. The comment on heapAddrBits explains that mmap on AIX returns addresses up to 0x0afffffffffffff, but if you can lower maxChunks artificially is that not really true? maxChunks is computed via heapAddrBits.

0x0afffffffffffff is the official limit. However, most of the processes don't need that much memory. As far as I remember, the highest address in a Go program that I have seen, was near 0x0a00040000000000. I think that's why I was able to lower maxChunks artificially. Note that at tip, it doesn't seem possible anymore...

Anyway, I've made some tests with different values for maxChunks to see how much performances are impacted. I've run them before the commit switching to the new page allocator (ie https://github.com/golang/go/commit/52d5e76b39a80a7e39754ff9564a7471551b3cdd)

maxChunks value time of go test -v -run=TestPageAlloc
0x0004000000 (Linux) 43s
0x0040000000 44s
0x0400000000 63s
0x4000000000 (default AIX) >600s

As you can see, performances are falling down after 0x0400000000.

I think it should be possible to have this value on AIX. Because if I understand correctly and please tell me if I'm completely wrong, you need a chunk for every possible address returned by mmap ie from 0x0a00000000000000 to 0x0afffffffffffff on AIX. To simplify, a chunk is allocated for every address starting from 0x0. As on AIX, the solution doesn't work and mmap addresses always have this '0x0a', don't you think that saying the first chunk deals with 0x0a00000000000000 instead of 0 is possible ? Therefore, maxChunks value would be 0x0400000000. Am I correct ? If yes, I can be a first workaround before finding a better solution.

I can write a couple of patches and if you have some time could you try them for me? Or if I had access to a machine (the gomote now is going to be really hard to get, due to slowness) I could dig deeper into debugging this myself.

I've created a special user for this kind of purpose on our build machine, just send me your SSH key by mail and I'll grant you the access.

Meanwhile, I'll continue to search for a solution.

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

The allocation is actually working pretty find. There is no freeze at this moment. The freeze occurs for once a PageAlloc test have ended (maybe during a GC phase or something like this).

Also, I noticed that sysMap on AIX unmaps memory and then remaps it, which may also be a problem here. Would implementing/using mprotect on AIX work better?

This unmmap is needed because AIX mmap doesn't work on an already mmap area, which is the case when doing sysReserve. So yes that's normal. No idea about mprotect, I will have to check and see what's going on.

Also, when you say reducing maxChunks is fixing the freezes, do you mean that it's also bringing back the original performance?

Yes, I've set the Linux value, it does work. (I haven't run all.bash though).

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

The problem comes from maxChunks constant in (s *pageAlloc) init. On linux/ppc64le, maxChunks=0x4000000. On aix/ppc64, it's 0x4000000000, resulting on mapping 0x200000000000, with sysReserve. Reducing maxChunks is fixing the freezes, I've tested manually. Therefore, is it possible to reduce it ? Or should we find a way to not sysReserve all the chunks at once ?

Helflym

comment created time in 3 months

issue commentgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

The problem might be larger than just this new page allocator. Its tests seem to pass correctly but a freeze (about 30s) occurs after each one, even with the old page allocator. It seems to loop inside runtime.sysmon. I'll continue to investigate.

go test -v -run=TestPageAllocScav
=== RUN   TestPageAllocScavenge
=== RUN   TestPageAllocScavenge/ScavMultiple
    TestPageAllocScavenge/ScavMultiple: mgcscavenge_test.go:368: start
    TestPageAllocScavenge/ScavMultiple: mgcscavenge_test.go:381: end

^\SIGQUIT: quit
PC=0x90000000056c270 m=2 sigcode=8

goroutine 0 [idle]:
runtime.usleep(0x271096ea48b0)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/os2_aix.go:523 +0x54 fp=0x110279500 sp=0x1102794b8 pc=0x100035694
runtime.sysmon()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:4453 +0xa4 fp=0x110279580 sp=0x110279500 pc=0x100047fc4
runtime.mstart1()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:1125 +0xf4 fp=0x1102795b8 sp=0x110279580 pc=0x10003eca4
runtime.mstart()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:1090 +0x5c fp=0x1102795e8 sp=0x1102795b8 pc=0x10003eb8c

goroutine 1 [chan receive]:
runtime.gopark(0x11007e800, 0xa00040000062298, 0xe170001000c6938, 0x2)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:304 +0x118 fp=0xa00040000083a00 sp=0xa000400000839d0 pc=0x10003c1e8
runtime.chanrecv(0xa00040000062240, 0xa00040000083b50, 0x100040000000180, 0x1000e40bc)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/chan.go:563 +0x334 fp=0xa00040000083aa0 sp=0xa00040000083a00 pc=0x100005f14
runtime.chanrecv1(0xa00040000062240, 0xa00040000083b50)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/chan.go:433 +0x24 fp=0xa00040000083ae0 sp=0xa00040000083aa0 pc=0x100005b84
testing.(*T).Run(0xa000400000ac120, 0x1001f2f45, 0x15, 0x11007ff80, 0x10000000000036c)
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:1006 +0x338 fp=0xa00040000083ba0 sp=0xa00040000083ae0 pc=0x1000e40d8
testing.runTests.func1(0xa000400000ac000)
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:1247 +0x78 fp=0xa00040000083c00 sp=0xa00040000083ba0 pc=0x1000e7f28
testing.tRunner(0xa000400000ac000, 0xa00040000083d28)
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:954 +0xe4 fp=0xa00040000083c60 sp=0xa00040000083c00 pc=0x1000e3d34
testing.runTests(0xa0004000000e0a0, 0x1101f2fc0, 0x13e, 0x13e, 0x0)
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:1245 +0x2d0 fp=0xa00040000083d48 sp=0xa00040000083c60 pc=0x1000e5590
testing.(*M).Run(0xa0004000000c080, 0x0)
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:1162 +0x188 fp=0xa00040000083e58 sp=0xa00040000083d48 pc=0x1000e4618
runtime_test.TestMain(0xa0004000000c080)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/crash_test.go:28 +0x2c fp=0xa00040000083eb8 sp=0xa00040000083e58 pc=0x100172b2c
main.main()
        _testmain.go:1152 +0x130 fp=0xa00040000083f50 sp=0xa00040000083eb8 pc=0x1001e93e0
runtime.main()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:203 +0x28c fp=0xa00040000083fc0 sp=0xa00040000083f50 pc=0x10003bd3c
runtime.goexit()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/asm_ppc64x.s:884 +0x4 fp=0xa00040000083fc0 sp=0xa00040000083fc0 pc=0x100072774

goroutine 2 [force gc (idle)]:
runtime.gopark(0x11007ead8, 0x1102044d0, 0x1114000000000000, 0x1)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:304 +0x118 fp=0xa00040000038780 sp=0xa00040000038750 pc=0x10003c1e8
runtime.goparkunlock(...)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:310
runtime.forcegchelper()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:253 +0xd0 fp=0xa000400000387c0 sp=0xa00040000038780 pc=0x10003c050
runtime.goexit()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/asm_ppc64x.s:884 +0x4 fp=0xa000400000387c0 sp=0xa000400000387c0 pc=0x100072774
created by runtime.init.5
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:242 +0x34

goroutine 3 [GC sweep wait]:
runtime.gopark(0x11007ead8, 0x110204920, 0xc14000000000000, 0x1)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:304 +0x118 fp=0xa0004000004bf78 sp=0xa0004000004bf48 pc=0x10003c1e8
runtime.goparkunlock(...)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:310
runtime.bgsweep(0xa00040000056000)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/mgcsweep.go:70 +0xac fp=0xa0004000004bfb8 sp=0xa0004000004bf78 pc=0x100026e2c
runtime.goexit()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/asm_ppc64x.s:884 +0x4 fp=0xa0004000004bfb8 sp=0xa0004000004bfb8 pc=0x100072774
created by runtime.gcenable
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/mgc.go:214 +0x58

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x11007ead8, 0x1102048e0, 0xd14000000000000, 0x1)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:304 +0x118 fp=0xa0004000004af10 sp=0xa0004000004aee0 pc=0x10003c1e8
runtime.goparkunlock(...)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:310
runtime.bgscavenge(0xa00040000056000)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/mgcscavenge.go:219 +0xf0 fp=0xa0004000004afb8 sp=0xa0004000004af10 pc=0x1000257f0
runtime.goexit()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/asm_ppc64x.s:884 +0x4 fp=0xa0004000004afb8 sp=0xa0004000004afb8 pc=0x100072774
created by runtime.gcenable
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/mgc.go:215 +0x78

goroutine 5 [finalizer wait]:
runtime.gopark(0x11007ead8, 0x11022ef48, 0x1014040000056000, 0x1)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:304 +0x118 fp=0xa00040000038f20 sp=0xa00040000038ef0 pc=0x10003c1e8
runtime.goparkunlock(...)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:310
runtime.runfinq()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/mfinal.go:175 +0xc8 fp=0xa00040000038fc0 sp=0xa00040000038f20 pc=0x10001aab8
runtime.goexit()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/asm_ppc64x.s:884 +0x4 fp=0xa00040000038fc0 sp=0xa00040000038fc0 pc=0x100072774
created by runtime.createfing
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/mfinal.go:156 +0x90

goroutine 6 [chan receive]:
runtime.gopark(0x11007e800, 0xa00040000062358, 0xe170001000c6938, 0x2)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/proc.go:304 +0x118 fp=0xa00040000085900 sp=0xa000400000858d0 pc=0x10003c1e8
runtime.chanrecv(0xa00040000062300, 0xa00040000085a50, 0x100040000001680, 0x1000e40bc)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/chan.go:563 +0x334 fp=0xa000400000859a0 sp=0xa00040000085900 pc=0x100005f14
runtime.chanrecv1(0xa00040000062300, 0xa00040000085a50)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/chan.go:433 +0x24 fp=0xa000400000859e0 sp=0xa000400000859a0 pc=0x100005b84
testing.(*T).Run(0xa000400000ad320, 0x1001ee2ae, 0xc, 0xa0004000004d4a0, 0xa00040000049e88)
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:1006 +0x338 fp=0xa00040000085aa0 sp=0xa000400000859e0 pc=0x1000e40d8
runtime_test.TestPageAllocScavenge(0xa000400000ac120)
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/mgcscavenge_test.go:383 +0x13cc fp=0xa00040000085f50 sp=0xa00040000085aa0 pc=0x1001a0ffc
testing.tRunner(0xa000400000ac120, 0x11007ff80)
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:954 +0xe4 fp=0xa00040000085fb0 sp=0xa00040000085f50 pc=0x1000e3d34
runtime.goexit()
        /opt/freeware/src/packages/BUILD/goroot/src/runtime/asm_ppc64x.s:884 +0x4 fp=0xa00040000085fb0 sp=0xa00040000085fb0 pc=0x100072774
created by testing.(*T).Run
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:1005 +0x31c

goroutine 7 [running]:
        goroutine running on other thread; stack unavailable
created by testing.(*T).Run
        /opt/freeware/src/packages/BUILD/goroot/src/testing/testing.go:1005 +0x31c

r0   0xffffffffffffffff r1   0x110279100
r2   0xffffffffffffffff r3   0xffffffffffffffff
r4   0xffffffffffffffff r5   0xffffffffffffffff
r6   0xffffffffffffffff r7   0xffffffffffffffff
r8   0xffffffffffffffff r9   0xffffffffffffffff
r10  0xffffffffffffffff r11  0xffffffffffffffff
r12  0xffffffffffffffff r13  0x110281800
r14  0x0        r15  0x0
r16  0x0        r17  0x0
r18  0x0        r19  0x0
r20  0x0        r21  0x0
r22  0x0        r23  0x0
r24  0x0        r25  0x0
r26  0x1102792a0        r27  0x0
r28  0xfffffffffffffe0  r29  0x9001000a0094210
r30  0x9001000a0004b40  r31  0x0
pc   0x90000000056c270  ctr  0xffffffff00000000
link 0xffffffffffffffff xer  0xffffffff
ccr  0x0        trap 0x0
exit status 2
FAIL    runtime 35.807s
Helflym

comment created time in 3 months

issue openedgolang/go

runtime: performance issue with the new page allocator on aix/ppc64

Since the new page allocator (#35112) has been enabled by default, the runtime is extremely slow. Some recent failures on aix/ppc64 builder seems also related, starting from CL 190622. There is a timeout during runtime tests (cf https://build.golang.org/log/7e68765a5fe5e9887ef06fd90de1c9ae6682e73d https://build.golang.org/log/d09de5259a97061169ce2648541666ba1101fc1c)

Before CL 201765:

$ time ./make.bash 
Building Go cmd/dist using /opt/freeware/lib/golang. (go1.13.4 aix/ppc64)
Building Go toolchain1 using /opt/freeware/lib/golang.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for aix/ppc64.
---
Installed Go for aix/ppc64 in /opt/freeware/src/packages/BUILD/goroot
Installed commands in /opt/freeware/src/packages/BUILD/goroot/bin

real    1m52.246s
user    2m53.752s
sys     0m16.242s

After

$ time ./make.bash 
Building Go cmd/dist using /opt/freeware/lib/golang. (go1.13.4 aix/ppc64)
Building Go toolchain1 using /opt/freeware/lib/golang.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for aix/ppc64.
---
Installed Go for aix/ppc64 in /opt/freeware/src/packages/BUILD/goroot
Installed commands in /opt/freeware/src/packages/BUILD/goroot/bin

real    20m44.516s
user    4m56.656s
sys     23m5.166s

cc @mknyszek

created time in 3 months

pull request commentolekukonko/ts

Add AIX support

Any news about the problems with Travis ?

Helflym

comment created time in 4 months

issue openedgolang/go

TestManyCalls is failing on aix/ppc64

aix/ppc64: https://build.golang.org/log/b623bdfa2f850fe357514464b30cb2b8af74964e

##### ../misc/cgo/testcarchive
--- FAIL: TestManyCalls (5.16s)
    carchive_test.go:861: unexpected fault address 0x100000000000000
        fatal error: fault
        [signal SIGSEGV: segmentation violation code=0x32 addr=0x100000000000000 pc=0x100000000000000]
        
        goroutine 17 [running, locked to thread]:
        runtime.throw(0x10005d20b, 0x5)
        	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/runtime/panic.go:1045 +0x68 fp=0xa00010000076ed0 sp=0xa00010000076e90 pc=0x100029128
        runtime: unexpected return pc for runtime.sigpanic called from 0x100000000000000
        stack: frame={sp:0xa00010000076ed0, fp:0xa00010000076f10} stack=[0xa00010000076000,0xa00010000077000)
        0a00010000076dd0:  000000010005d0c5  0000000000000001 
        0a00010000076de0:  0000000000000001  0000000000000000 
        0a00010000076df0:  000000010002ac38 <runtime.printstring+104>  0000000000000002 
        0a00010000076e00:  0000000000000002  0a00010000076ebb 
        0a00010000076e10:  0000000000000002  000000010005d0c5 
        0a00010000076e20:  0000000100000001  00000001000292c8 <runtime.fatalthrow+72> 
        0a00010000076e30:  000000010003e540 <runtime.sigpanic+992>  0000000000000000 
        0a00010000076e40:  000000010002a528 <runtime.printnl+56>  0000000100029128 <runtime.throw+104> 
        0a00010000076e50:  0a00010000076ebb  0000000000000011 
        0a00010000076e60:  000000010005d0c5  0a00010000076e70 
        0a00010000076e70:  0000000100051ce0 <runtime.fatalthrow.func1+0>  0a00010000000600 
        0a00010000076e80:  0000000100029128 <runtime.throw+104>  0a00010000076e90 
        0a00010000076e90:  000000010003e560 <runtime.sigpanic+1024>  0000000000000001 
        0a00010000076ea0:  0a00010000000600  000000010003e548 <runtime.sigpanic+1000> 
        0a00010000076eb0:  0a00010000076eb8  0000000100051c60 <runtime.throw.func1+0> 
        0a00010000076ec0:  000000010005d20b  0000000000000005 
        0a00010000076ed0: <0100000000000000  00000001000523e0 <runtime.exitsyscallfast.func1+0> 
        0a00010000076ee0:  0100010000076ec8  0000000000000000 
        0a00010000076ef0:  000000010005d20b  0000000000000005 
        0a00010000076f00:  0000000000000000  0100000000000000 
        0a00010000076f10: >0100000000000000  0a00010000076ee0 
        0a00010000076f20:  000000011001da18  0000000000000000 
        0a00010000076f30:  0000000000000000  00000001000554a8 <runtime.cgocallback_gofunc+136> 
        0a00010000076f40:  0000000000000000  0000000000000000 
        0a00010000076f50:  0000000000000000  0000000000000000 
        0a00010000076f60:  0000000000000000  00000001000555e4 <runtime.goexit+4> 
        0a00010000076f70:  0000000000000000  0000000000000000 
        0a00010000076f80:  0000000000000000  0000000000000000 
        0a00010000076f90:  0000000000000000  0000000000000000 
        0a00010000076fa0:  0a00010000076fe0  0a00010000000600 
        0a00010000076fb0:  00000001000555e4 <runtime.goexit+4>  0000000000000000 
        0a00010000076fc0:  0000000000000000  0000000000000000 
        0a00010000076fd0:  0000000000000000  0000000000000000 
        0a00010000076fe0:  0000000000000000  0000000000000000 
        0a00010000076ff0:  0000000000000000  0000000000000000 
        runtime.sigpanic()
        	/ramdisk8GB/workdir-host-aix-ppc64-osuosl/go/src/runtime/signal_unix.go:601 +0x400 fp=0xa00010000076f10 sp=0xa00010000076ed0 pc=0x10003e560
        
    carchive_test.go:862: signal: IOT/Abort trap (core dumped)

It seems to be an hidden bug which has been triggered by CL 204957, but it's not related. In first glance, the stack before runtime.cgocallback seems wrong. I'm working on it.

created time in 4 months

pull request commentolekukonko/ts

Add AIX support

Weird, I haven't changed anything with linux/amd64. Moreover, it's working on my Linux without any problem. Are you sure your travis is still working as wanted ?

Helflym

comment created time in 4 months

PR opened olekukonko/ts

Add AIX support

Like Solaris, AIX can't call syscall.Syscall and need to use sys/unix functions

+30 -21

0 comment

5 changed files

pr created time in 4 months

push eventHelflym/ts

Clément Chigot

commit sha 6ada20b383cd50d48dab7c11720dacf86f7ee714

Add AIX support Like Solaris, AIX can't call syscall.Syscall and need to use sys/unix functions

view details

push time in 4 months

fork Helflym/ts

Simple go Application to get Terminal Size

fork in 4 months

issue commentgolang/go

runtime: TestNetpollBreak, TestNonblockingPipe, TestSetNonblock failing on aix-ppc64

My bad, that was the wrong goto ! This one is problematic.

bcmills

comment created time in 4 months

issue commentgolang/go

runtime: TestNetpollBreak, TestNonblockingPipe, TestSetNonblock failing on aix-ppc64

TestSetNonblock and TestNonblockingPipe are failing because syscall.Syscall isn't available on AIX. I've made the patch but I've a problem with my google account. I'll push it asap. TestNetpollBreak is a bit more complicated. It seems to be related to goto retry at line 176. But I've to be sure we can remove it without triggering any problems.

bcmills

comment created time in 4 months

issue commentgolang/go

cmd: calling C code without cgo on aix/ppc64

The test is actually working because it's only checking the compilation. The program made isn't run. So nothing to do at the moment.

Helflym

comment created time in 4 months

issue commentgolang/go

aix: builder is offline

It's ok now. The folder where builds are made, workdir-host-aix-ppc64-osuosl, seems to have been removed by something.

bcmills

comment created time in 4 months

issue openedgolang/go

cmd: calling C code without cgo on aix/ppc64

In CL 198798, @jpap allows Go asm to call/jump directly to a C function defined in a .syso files. The jump is performed like this JMP Cfunc(SB). However, on AIX, this JMP doesn't work. A C function has two symbols:

  • Cfunc, a data symbol with a pointer to .Cfunc and a pointer to the TOC symbol (something similar to GOT stuff with ELF format)
  • .Cfunc, a text symbol which the label in .text section.

Therefore, JMP Cfunc(SB) is jumping to a data symbol which obviously doesn't work. The normal 'AIX' code should be JMP .Cfunc(SB). However, there are several problems with that:

  • GO asm doesn't seem able to parse this "." function.
  • Special AIX code will be needed for every programs using this stuff.

What's the policy of Go in this case ? Do we want to match C code (ie using JMP .Cfunc(SB)) or do we want to be closer to Go code (ie using JMP Cfunc(SB)) and afterwards changing ASM generated in cmd/internal.

Note that both solution might be added. So people might want to write special ASM code for AIX with "dot" function, etc. While others just want their code to work in all platforms.

created time in 4 months

push eventHelflym/flock

Clément Chigot

commit sha 63888f6eb216fe8a698af9de1eb9f7c2d5b0dbc4

Update flock_aix.go

view details

push time in 4 months

issue commentgolang/go

cmd/compile: program that requires zerorange of >=32 bytes fails to compile on aix/ppc64

I've somehow missed the part with ADUFFZERO when I've adapted rewriteToUseGot into rewriteToUseTOC. I'll send a CL in order to fix it. Note that it only appears now (almost a year after the implementation) that the ADUFFZERO instruction wasn't correctly handled. It seems that there is no tests at all, generating at least one ADUFFZERO. Is this the case or they are somehow disabled on aix/ppc64 ?

danscales

comment created time in 5 months

issue commentgolang/go

cmd/go: new internal/goroot test broke aix, illumos, solaris builders

I think this is linked with CL 196237. Symbol runtime.setmodinfo is only available in gcc-master with go1.13. In previous gcc version thus previous Go version, runtime/debug.setmodinfo is still linked to runtime..z2fdebug.setmodinfo.

gcc-9.1.0:

go/runtime/debug/mod.go://go:linkname setmodinfo runtime..z2fdebug.setmodinfo

gcc-svn:

go/runtime/debug.go://go:linkname setmodinfo runtime.setmodinfo

Thus, gccgo version must be checked when creating _gomod_.go.

AIX builder is still in gcc 8 (for various reasons) and I think it's the same for Illumos and Solaris builders.

ALTree

comment created time in 5 months

fork Helflym/color

Color package for Go (golang)

http://godoc.org/github.com/fatih/color

fork in 5 months

more