profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/penberg/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Pekka Enberg penberg Finland https://penberg.org Software engineer interested in distributed systems, operating systems, and cloud computing.

manticoreos/manticore 240

Manticore is a research operating system, written in Rust.

penberg/fjord 198

Fjord, F# programming language for the JVM.

penberg/linux-kvm 155

Native Linux KVM tool

penberg/falcon 140

Falcon, the open source ultra low-latency FIX engine for Java

penberg/helix 104

Helix, a market data feed handler for C and C++.

penberg/hornet 55

Hornet, a JVM optimized for low-latency applications.

penberg/go-osxhv 34

Go bindings for the OS X Hypervisor.framework API

penberg/libtrading 33

Libtrading, an ultra low-latency trading connectivity library for C and C++.

penberg/godoom 16

A Doom clone written in OpenGL and Go.

penberg/linux-networking 9

Notes on Linux network internals

starteddenoland/deno

started time in 4 hours

push eventpenberg/manticore-dev

Pekka Enberg

commit sha 679b54c2b8d721f9af8a136789e7d52295e9acae

Add "run-test" script This automates the step of creating an ISO and running it.

view details

Pekka Enberg

commit sha dab38ca3b3f2a7a73017d5a1dc2eb632c514aa50

Fix compatibility with Rust 1.57 nightly

view details

Pekka Enberg

commit sha 0c74866eeab2d1efdbd488019c758367f9946514

install-toolchain: Update to Rust 1.57

view details

Pekka Enberg

commit sha 81cb3b54800e8a0e1f77bee0634ae82855b7e1c1

install-toolchain: Fix Rust nightly version

view details

Pekka Enberg

commit sha 3cfbcab6811a86009c3aab8d4da676187606ee82

liblinux: Move assert() failure out-of-line Move assert() failure case to out-of-line __assert_fail() function to ensure types passed to printf() format are correct. Fixes the following GCC complaint: In file included from /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:1: /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c: In function ‘main’: /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: ' ' flag used with ‘%x’ gnu_printf format [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~ /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: format ‘%x’ expects a matching ‘unsigned int’ argument [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~

view details

Pekka Enberg

commit sha bf5d09a3ac02ad49ef36a007806d2569168e712c

kernel: Add spin lock

view details

Pekka Enberg

commit sha 3209a4dea104a6810263278084ef860a7bdcdd8d

kernel: Make printf() SMP-safe

view details

Pekka Enberg

commit sha d07f2f310f1319a025daab0c634435e1c378936f

x86/apic: Inter-process interrupt (IPI) support

view details

Pekka Enberg

commit sha 2a3611da1d504dde01288d58c5e410359c41f9cd

x86/apic: Add apic_ipi_allbutself() helper

view details

Pekka Enberg

commit sha cebd04f303d4a16ea6c3e5ac78580bcc33fb7f19

x86/apic: Add apic_local_id() helper

view details

Pekka Enberg

commit sha e41393708b7444d36ae857efc5ddd171d0019a1a

x86/smp: SMP support

view details

Pekka Enberg

commit sha 027ee4d396218d587683e7803f33be6b57d0ac21

arch/x86: Setup APIC before platform config parsing

view details

Pekka Enberg

commit sha 2b470e9954031ac2d9bd880911c1737844d8b934

x86/acpi: Parse CPU local ID

view details

Pekka Enberg

commit sha 5c5d907d0049c1ddc95d3fc3c54e261343c7e156

x86/cpuid: X86_CPUID_FEATURE_EBX_APIC_ID_SHIFT

view details

Pekka Enberg

commit sha 55cc3c02f1238e7c9e569d4dac807792ad50511f

x86/boot: Secondary CPU start up

view details

Pekka Enberg

commit sha 8a0ea72622120cffad68678e3de5cdad8757eb77

x86/setup: SMP initialization

view details

Pekka Enberg

commit sha 4f3c330a28bbe1d03158569e06f8e44b893e0dbd

kernel/init: secondary bringup

view details

push time in 6 days

push eventpenberg/manticore-dev

Pekka Enberg

commit sha dab38ca3b3f2a7a73017d5a1dc2eb632c514aa50

Fix compatibility with Rust 1.57 nightly

view details

Pekka Enberg

commit sha 0c74866eeab2d1efdbd488019c758367f9946514

install-toolchain: Update to Rust 1.57

view details

Pekka Enberg

commit sha 81cb3b54800e8a0e1f77bee0634ae82855b7e1c1

install-toolchain: Fix Rust nightly version

view details

Pekka Enberg

commit sha 3cfbcab6811a86009c3aab8d4da676187606ee82

liblinux: Move assert() failure out-of-line Move assert() failure case to out-of-line __assert_fail() function to ensure types passed to printf() format are correct. Fixes the following GCC complaint: In file included from /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:1: /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c: In function ‘main’: /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: ' ' flag used with ‘%x’ gnu_printf format [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~ /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: format ‘%x’ expects a matching ‘unsigned int’ argument [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~

view details

Pekka Enberg

commit sha 016a05fe467ccd0a446d528d65cb8c66b707f492

liblinux: POSIX threads

view details

push time in 6 days

push eventmanticoreos/manticore

Pekka Enberg

commit sha 3cfbcab6811a86009c3aab8d4da676187606ee82

liblinux: Move assert() failure out-of-line Move assert() failure case to out-of-line __assert_fail() function to ensure types passed to printf() format are correct. Fixes the following GCC complaint: In file included from /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:1: /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c: In function ‘main’: /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: ' ' flag used with ‘%x’ gnu_printf format [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~ /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: format ‘%x’ expects a matching ‘unsigned int’ argument [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~

view details

push time in 6 days

push eventpenberg/manticore-dev

Pekka Enberg

commit sha 3cfbcab6811a86009c3aab8d4da676187606ee82

liblinux: Move assert() failure out-of-line Move assert() failure case to out-of-line __assert_fail() function to ensure types passed to printf() format are correct. Fixes the following GCC complaint: In file included from /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:1: /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c: In function ‘main’: /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: ' ' flag used with ‘%x’ gnu_printf format [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~ /home/penberg/src/penberg/manticore-dev/usr/liblinux/include/assert.h:10:41: warning: format ‘%x’ expects a matching ‘unsigned int’ argument [-Wformat=] 10 | fprintf(stderr, "%s:%d: %s: Assertion `" #expr "' failed.\n", __FILE__, __LINE__, __func__); \ | ^~~~~~~~~~~~~~~~~~~~~~~~ /home/penberg/src/penberg/manticore-dev/usr/tests/tst-vmspace_alloc.c:21:17: note: in expansion of macro ‘assert’ 21 | assert(p[i] == (i % 0xff)); | ^~~~~~

view details

push time in 6 days

push eventmanticoreos/manticore

Pekka Enberg

commit sha 81cb3b54800e8a0e1f77bee0634ae82855b7e1c1

install-toolchain: Fix Rust nightly version

view details

push time in 6 days

push eventpenberg/manticore-dev

Pekka Enberg

commit sha 81cb3b54800e8a0e1f77bee0634ae82855b7e1c1

install-toolchain: Fix Rust nightly version

view details

push time in 6 days

push eventmanticoreos/manticore

Pekka Enberg

commit sha 0c74866eeab2d1efdbd488019c758367f9946514

install-toolchain: Update to Rust 1.57

view details

push time in 6 days

push eventpenberg/manticore-dev

Pekka Enberg

commit sha dab38ca3b3f2a7a73017d5a1dc2eb632c514aa50

Fix compatibility with Rust 1.57 nightly

view details

Pekka Enberg

commit sha 0c74866eeab2d1efdbd488019c758367f9946514

install-toolchain: Update to Rust 1.57

view details

push time in 6 days

PR closed manticoreos/manticore

Added documentation

A lot more documentation can be added; just the initial cut. Fixes https://github.com/manticoreos/manticore/issues/29.

  • My fork docs branch: https://manticoredocumentation.readthedocs.io/en/docs/

Screen Shot 2019-08-30 at 2 39 22 AM

  • Main docs: https://manticoredocumentation.readthedocs.io
+623 -0

1 comment

31 changed files

tapaswenipathak

pr closed time in 6 days

push eventmanticoreos/manticore

Pekka Enberg

commit sha dab38ca3b3f2a7a73017d5a1dc2eb632c514aa50

Fix compatibility with Rust 1.57 nightly

view details

push time in 6 days

PullRequestReviewEvent

pull request commentscylladb/scylla

dist/redhat: add scylla-python3 to scylla metapackage dependency

@syuu1228 @avikivity I actually see this dependency in 2021.1 scylla.spec too:

Requires:       %{product}-server = %{version} %{product}-conf = %{version} %{product}-python3 = %{version} %{product}-kernel-conf = %{version} %{product}-jmx = %{version} %{product}-tools = %{version} %{product}-tools-core = %{version}

What am I missing @syuu1228?

syuu1228

comment created time in 11 days

push eventscylladb/scylla

Felipe Mendes

commit sha 1b8dff63c3993deb527dae1b3912206ac4491901

iotune - Fix i3en.xlarge check i3en.xlarge is currently not getting tuned properly. A quick test using Scylla AMI ( ami-07a31481e4394d346 ) reveals that the storage capabilities under this instance are greatly reduced: $ grep iops /etc/scylla.d/io_properties.yaml read_iops: 257024 write_iops: 174080 This patch corrects this typo, in such a way that iotune now properly tunes this instance type. Closes #9298

view details

push time in 18 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventscylladb/scylla-machine-image

Takuya ASADA

commit sha 3c49e6df1a9224a063dde5e60e4e49ae976528a0

move CPU scaling setup from scylla_image_setup to scylla_install_image Currently, CPU scaling setup is mistakenly running in AMI building time. This is not correct because the AMI image can run other instance type. To fix this issue, disable CPU scaling setup on scylla_image_setup, and add it to scylla_install_image. Fixes scylladb/scylla#9273

view details

push time in 19 days

PR merged scylladb/scylla-machine-image

Reviewers
move CPU scaling setup from scylla_image_setup to scylla_install_image

Currently, CPU scaling setup is mistakenly running in AMI building time. This is not correct because the AMI image can run other instance type. To fix this issue, disable CPU scaling setup on scylla_image_setup, and add it to scylla_install_image.

Fixes scylladb/scylla#9273

+2 -1

1 comment

2 changed files

syuu1228

pr closed time in 19 days

issue closedscylladb/scylla

Scylla 4.2.2 governor dropped from 'performance' to 'schedutil'

The governor has dropped from 'performance' to 'schedutil'. In one of my benchmarks I saw 10% throughput increase by upgrading the governor from 'schedutil' to 'performance'.

After doing a binary search over the Scylla versions, the Scylla 4.2.2 tag is the first one that suffering from this problem.

In 4.2.1 there is a tuned.service enabled and in 4.2.2 there is not.

After disabling the tuned.service on 4.2.1 and rebooting, the instance also is running with the 'schedutil' governor. So it seems that the 'tuned' part is responsible for starting the performance governor.

One of the changes in 4.2.2 is in the file dist/redhat/scylla.spec which has the following 2 extra lines:

scylla.spec:# tuned overwrites our sysctl settings
scylla.spec:Obsoletes:  tuned

So here the tuned functionality is disabled.

This problem is confirmed on the centos AWS images; the Ubuntu based images like 2021.1 and 4.5 rc do not suffer from this problem.

Since the problem doesn't seem to depend on the cloud provider, I guess that redhat based instances in other cloud providers could suffer from the same problem.

Workarounds

Either call:

sudo cpupower frequency-set -g governor

Or if you want to make it durable:

sudo systemctl enable cpupower.service

Update

The scylla_cpuscaling_setup script should take care of setting the governor to performance. Either it doesn't get called, or somehow there is a failure when it is called. The /etc/sysconfig/cpupower file exists with the following content:

# See 'cpupower help' and cpupower(1) for more info
CPUPOWER_START_OPTS="frequency-set -g performance"
CPUPOWER_STOP_OPTS="frequency-set -g ondemand"

The problem is that the cpupower.service isn't started.

In the journalctl I do not see the following call being made:

Created symlink from /etc/systemd/system/multi-user.target.wants/cpupower.service to /usr/lib/systemd/system/cpupower.service.

When calling the scylla_cpuscaling_setup script manually, the cpupower.service is created and the performance governor mode is set. It seems that the script is working fine, so the plausible explanation is that the script isn't called. Another indication that the script has not run is that the 'kernel-tools' package isn't installed.

closed time in 19 days

pveentjer

startedakiradeveloper/lol

started time in 20 days

issue commentscylladb/scylla

Scylla 4.2.2 governor dropped from 'performance' to 'schedutil'

@pveentjer So commit 615b8e81844e47fd54e6043d1ef0a746aa9348d3 caused a regression of CPU governor not being set to performance on RPM-based systems. Did I understand this correctly? You don't see the issue with the current AMI because it's Ubuntu-based, but the regression still exists in RPM systems?

/cc @avikivity @syuu1228

pveentjer

comment created time in 24 days

push eventscylladb/scylla

Beni Peled

commit sha 4fe4aa190d3b2d0ab3d45f0e247e08b890c6c32e

dist-check: add podman support ...and use container term instead of docker Closes #9265

view details

push time in 25 days

pull request commentscylladb/scylla-python3

reloc: clean up '.orig' temporary directory before building deb package

@syuu1228 When you say "sent a new PR", do you mean there's another pull request (which one) or did you update this one (I see a force push)?

syuu1228

comment created time in 25 days

issue commentscylladb/scylla-machine-image

Test failed because of 'elrepo' repository URL has been changed for CentOS7

The following steps unblock the upgrade for me:

sudo rm /etc/yum.repos.d/elrepo-archive.repo
sudo yum install https://www.elrepo.org/elrepo-release-7.el7.elrepo.noarch.rpm

@syuu1228, can you please confirm that this is the kind of workaround you were thinking of?

juliayakovlev

comment created time in a month

issue commentscylladb/scylla-machine-image

Enable image to work with only the root disk for AWS t2 instances

@syuu1228 Let's make the t1 instances properly supported by the AMI then.

slivne

comment created time in a month

issue commentscylladb/scylla

Failed to install scylla on Suse Enterprise 15 for jre-1.8.0-headless dependency

@amoskong SUSE is not an officially supported platform so there's no installation guide for it either. You could add support for SUSE in the Scylla installer, though: https://github.com/scylladb/scylla-web-install

amoskong

comment created time in a month

PullRequestReviewEvent

push eventscylladb/scylla

Nadav Har'El

commit sha d7474ddff3d5b49f03848fccdc78ad758e95ab4c

dist/docker: fix errors in README.md The (oddly-placed) document dist/docker/debian/README.md explains how a developer can build a Scylla docker image using a self-built Scylla executable. While the document begins by saying that you can "build your own Scylla in whatever build mode you prefer, e.g., dev.", the rest of the instructions don't fit this example mode "dev" - the second command does "ninja dist-deb" which builds *all* modes, while the third command forgets to pass the mode at all (and therefore defaults to "release"). The forth command doesn't work at all, and became irrelevant during a recent rewrite in commit e96ff3d. This patch modifies the document to fix those problems. It ends with an example of how to run the resulting docker image (this is usually the purpose of building a docker image - to run it and test it). I did this example using podman because I couldn't get it to work in docker. Later we can hopefully add the corresponding docker example. Fixes #9263. Signed-off-by: Nadav Har'El <nyh@scylladb.com> Message-Id: <20210829182608.355748-1-nyh@scylladb.com>

view details

push time in a month

push eventscylladb/scylla

Nadav Har'El

commit sha ed7106ebd7234574c38c7c1a7eaba2d170da7803

docker: fix regression of docker image ignoring command-line arguments Our docker image accepts various command-line arguments and translates them into Scylla arguments. For example, Alternator's getting-started document has the following example: ``` docker run --name scylla -d -p 8000:8000 scylladb/scylla-nightly:latest --alternator-port=8000 --alternator-write-isolation=always``` Recently, this stopped working and the extra arguments at the end were just ignored. It turns out that this is a regression caused by commit e96ff3d82d48c41f428e6fb2f622454da0c36cdf that changed our docker image creation process from Dockerfile to buildah. While the entry point specified in Dockerfile was a string, the same string in buildah has a strange meaning (an entry point which can't take arguments) and to get the original meaning, the entry point needs to be a JSON array. This is kind-of explained in https://github.com/containers/buildah/issues/732. So changing the entry point from a string to a JSON array fixes the regression, and we can again pass arguments to Scylla's docker image. Fixes #9247. Signed-off-by: Nadav Har'El <nyh@scylladb.com> Message-Id: <20210829180328.354109-1-nyh@scylladb.com>

view details

push time in a month

PullRequestReviewEvent