profile
viewpoint
Geoffrey Irving girving London https://naml.us Safety Researcher at @deepmind. AI will probably be wonderful; let's make that even more probable.

eddysystems/eddy 51

eddy - autocorrect for java

girving/duck 9

a functional language built around overloading

girving/kalah 4

A perfect kalah player for up to 5 stones per bin

distillpub/post--safety-needs-social-scientists 3

AI safety needs social scientists

girving/deepmath 2

Experiments towards neural network theorem proving

girving/games 2

Combinatorial game theory exploration code

girving/meld 2

Fork of http://git.gnome.org/cgit/meld

eddysystems/issues 1

Go to https://github.com/eddysystems/eddy instead

eddysystems/quibble 1

Neural network code reformatting

girving/banana 1

An automated Bananagram player

push eventgirving/pentago

Geoffrey Irving

commit sha 1674cd6bdf18a99b3db562c0663371d305029059

web/client/unit.js: Support for running specific tests

view details

Geoffrey Irving

commit sha 645382c1e5af19c42582a654e6cdc3f327bcb51a

Remove most tables from wasm The WebAssembly build now depends only on win_contributions, rotations, and reflections, which together take up only 19KB. This shrinks mid.wasm from 450,525 to 47,908 bytes. This required making non-table-based versions of various functions. Most such as pack/unpack were called few times, and sufficiently fast versions were easy. The hard function was halfsuper_wins, which went from depending only the very large superwin_info table to the much smaller win_contribution table, but got significantly slower as a result. Fortunately, I noticed that midengine was calling halfsuper_wins more times than necessary, and better caching made the final version faster than before this commit: 11.4 s to 10.9 s for `unit.js mid`.

view details

push time in 13 days

issue commentgoogle/jax

Adding `remat` causes an error to otherwise working code

Thanks for the fix and the interesting commentary! Also, evidence that you have well designed software: fixing a subtle bug by reducing the total amount of code. :)

tomhennigan

comment created time in 16 days

issue commentgoogle/jax

Adding `remat` causes an error to otherwise working code

@mattjj Does this reproduce for you?

tomhennigan

comment created time in 16 days

push eventgirving/pentago

Geoffrey Irving

commit sha 43d8b21bc25333ec32139a5d0e866c5ee832d86d

Use brfs and webworkify to inline worker+wasm into client.js Now we have a single Javascript file. It's 1M, though, and could use some shrinking at some point. In any case, since it's a single file the page should load faster; previously we had many sequential steps: 1. client.js loads 2. client.js runs, requests mid_worker.js 3. mid_worker.js loads 4. mid_worker.js runs, requests mid.wasm 5. mid.wasm loads Normal page loads needed to get through step 4 to do anything, since the mid_worker.js request was sequential. Now there's just one load.

view details

push time in 19 days

issue commentgoogle/jax

Add support for buffer donation (input/output aliasing)

@ibab: @tomhennigan is already on this.

hawkinsp

comment created time in 20 days

issue commentgoogle/jax

Should ShapeDtypeStruct.size exist?

NamedTuples are bad:

shape, dtype = struct  # WAT
girving

comment created time in 21 days

issue openedgoogle/jax

Should ShapeDtypeStruct.size exist?

ShapeDtypeStruct currently has dtype and shape. We could add numpy's size property via

class ShapeDtypeStruct(object):
  __slots__ = ["shape", "dtype"]
  def __init__(self, shape, dtype):
    self.shape = shape
    self.dtype = dtype

  @property
  def size(self):
    return np.prod(self.shape, dtype=int)

Do you want this patch?

created time in 21 days

issue commentgoogle/jax

It would be nice if jax.device_get returned numpy scalars

The printing point is ergonomics: Python things that look like integers normally print as bare integers (7, say), while Jax integers print like array(7, dtype=int32). And if you want to extract the normal ergonomic integer you have to do something awkward like x[()].

girving

comment created time in 21 days

issue commentgoogle/jax

It would be nice if jax.device_get returned numpy scalars

Basically numpy's default is scalars, so it wants to be loud if you return something nonstandard. And jax is returning something nonstandard, so the result is loud. The argument about redesigning numpy doesn't really apply in that sense given that numpy is fixed.

girving

comment created time in 21 days

issue commentgoogle/jax

It would be nice if jax.device_get returned numpy scalars

array("A good chunk of me wanting this is making simple numbers print as simple numbers, so that I can find outputs easier to read.", dtype=object)

girving

comment created time in 21 days

issue openedgoogle/jax

It would be nice if jax.device_get(jnp.asarray(7)) returned a numpy scalar

The following behavior is not quite idiomatic numpy:

In [5]: jax.device_get(jnp.abs(7))
Out[5]: array(7, dtype=int32)

Ideally, this would instead print 7. This is what would happen if rank 0 jax arrays converted to numpy scalars in jax.device_get, as is the default behavior in both numpy and tensorflow.

Is it possible to fix jax.device_get to make numpy scalars?

created time in 22 days

create barnchgirving/pentago

branch : synced/git-annex

created branch time in a month

create barnchgirving/pentago

branch : synced/master

created branch time in a month

push eventgirving/pentago

Geoffrey Irving

commit sha ed98ccaa05b8a9b97d0a5c1e2eade01ca9c6bd0b

values.js: Switch to us-central1 bucket

view details

push time in a month

push eventgirving/pentago

Geoffrey Irving

commit sha d8fe8a71985fbe094083284b910624ec29e66323

web/server: chmod -x games.js index.js

view details

push time in a month

push eventgirving/pentago

Geoffrey Irving

commit sha f6dbecc898b13ebf529b9549d8ba1a7758dacc3a

web/server: Remove obsolete dependencies (greenlock and redirect-https)

view details

push time in a month

push eventgirving/pentago

Geoffrey Irving

commit sha fb0c664f4015027a4c28d845abb25de76f9fcdba

Switch server from Rackspace to Google Cloud The server is now a Google Cloud Function, which is much easier to deploy. For this purpose, the server is also pure Javascript: its remaining C++ dependencies have been ported across.

view details

push time in a month

push eventgirving/pentago

Geoffrey Irving

commit sha eac2e80a7c7fa81ba68a25190ab9f0d9a136ee6c

web/misc: Hash computation on the Google side Everything matches up

view details

Geoffrey Irving

commit sha 2f49b63212e671377b11c5d237414cef38529549

web/server: Rackspace → Google I think a Cloud Function is better, so I'm going to switch to that next rather than deploy the server unmodified

view details

Geoffrey Irving

commit sha 8deb811df410b0306082832c038f96e2220abb2a

.dockerignore: Ignore some more stuff

view details

push time in a month

push eventgirving/pentago

Geoffrey Irving

commit sha ee212d3e6c82b0cbfff08ce6c269f7416e8cbf46

Scripts for getting data safely off Rackspace

view details

push time in a month

create barnchgirving/pentago

branch : irving-fastmid

created branch time in 2 months

create barnchgirving/pentago

branch : irving-progress

created branch time in 2 months

delete branch girving/pentago

delete branch : neural

delete time in 2 months

delete branch girving/pentago

delete branch : synced/git-annex

delete time in 2 months

delete branch girving/pentago

delete branch : synced/master

delete time in 2 months

delete branch girving/pentago

delete branch : bazel

delete time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha c606235d8eef14484e657023eab8329a43578031

midengine: Simplify allocation 1. Allocate all_wins1 on the stack. 2. Make workspace typed and aligned throughout now that Javascript doesn't play a role.

view details

Geoffrey Irving

commit sha 2f03ecfe0e54986302b9612308128db44dc2db45

midengine: Simplify away white_roto and black_root variables

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha 61641e6f639069e1dbbd31d78593f1ebbff844ca

midengine: Uniformize boundary condition

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha 2929b5c8cbd768e9f3c0d3e74788f0664e851893

web/cache: Test local_lru Required a bunch of test-related code, but more importantly inspired me to switch from Date.now() to manual time stamps.

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha e7b127aae3de0f7d24f7b2d73c8885ee107ad767

web/client: Add persistent caching This uses Window.localStorage where possible.

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha b92624ecf49e431ae48552738d353a2abb0b2cd1

Strip midgame solving out of server Now that the client does this by itself, we can use a smaller server.

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha dc13a155f4132fbec26d75aef0d7282ef0df61cc

Solve midgames and on locally via bare WebAssembly The pentago javascript client now solves boards with 18 or more stones locally via bare WebAssembly code running in a WebWorker. Bare WebAssembly means clang-only, without Emscripten. That in turn means no standard library, no malloc, etc. The midengine was already pretty light on memory allocation, so the main hardship was getting a stripped version of libc++ (for type_traits and similar niceties) that works in -ffreestanding mode without malloc and such. I did this by copying the libc++ headers into third_party/freestanding and adding a very large number of ifdefs. A wrinkle: the midengine is memory heavy, and WebAssembly instances can't ever shrink their overall memory size. To get around this, the client makes a fresh WebAssembly instance for each request and throws it away when complete. Thankfully the API separates compilation from instance creation, so this seems fine. Timings for an 18 stone board (https://perfect-pentago.net/#274440791932540184): MacBook Pro, 13-inch, 2019, 2.5 GHz Quad-Core Intel Core i5: Chrome 79 = 12.381 s Firefox 71 = 10.587 s Safari 13.0.4 = 12.572 s iPhone 11 Pro, iOS 13.3: 9.0 s (!) Other improvements along the way: 1. Remove the weird "boards" argument to midengine routines. All available boards are now computed inside the routines. 2. Reduce excessive templatization of midengine to help code size. This results in a small but acceptable 10-15% slowdown. 3. Make midengine work without SSE. 4. Make high_board_t smaller (16 -> 8 bytes) and leaner (no memory allocation). 5. Upgrade server to node 13.5.0. Extensive and painful binding tweaks required. 6. Use fetch instead of XMLHttpRequest in client. Posts which gave me the idea that bare WebAssembly was possible: * https://dassur.ma/things/c-to-webassembly * https://medium.com/@dougschaefer/going-straight-to-clang-for-webassembly-928df1484430

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha 0700e0faaf31abcd7c6b67dff14478df120177dc

Solve midgames and on locally via bare WebAssembly The pentago javascript client now solves boards with 18 or more stones locally via bare WebAssembly code running in a WebWorker. Bare WebAssembly means clang-only, without Emscripten. That in turn means no standard library, no malloc, etc. The midengine was already pretty light on memory allocation, so the main hardship was getting a stripped version of libc++ (for type_traits and similar niceties) that works in -ffreestanding mode without malloc and such. I did this by copying the libc++ headers into third_party/freestanding and adding a very large number of ifdefs. A wrinkle: the midengine is memory heavy, and WebAssembly instances can't ever shrink their overall memory size. To get around this, the client makes a fresh WebAssembly instance for each request and throws it away when complete. Thankfully the API separates compilation from instance creation, so this seems fine. Timings for an 18 stone board (https://perfect-pentago.net/#274440791932540184): MacBook Pro, 13-inch, 2019, 2.5 GHz Quad-Core Intel Core i5: Chrome 79 = 12.381 s Firefox 71 = 10.587 s Safari 13.0.4 = 12.572 s iPhone 11 Pro, iOS 13.3: 9.0 s (!) Other improvements along the way: 1. Remove the weird "boards" argument to midengine routines. All available boards are now computed inside the routines. 2. Reduce excessive templatization of midengine to help code size. This results in a small but acceptable 10-15% slowdown. 3. Make midengine work without SSE. 4. Make high_board_t smaller (16 -> 8 bytes) and leaner (no memory allocation). 5. Upgrade server to node 13.5.0. Extensive and painful binding tweaks required. 6. Use fetch instead of XMLHttpRequest in client. Posts which gave me the idea that bare WebAssembly was possible: * https://dassur.ma/things/c-to-webassembly * https://medium.com/@dougschaefer/going-straight-to-clang-for-webassembly-928df1484430

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha d9549312877cfa3175d7b765e348f59f3d18e830

Solve midgames and on locally via bare WebAssembly The pentago javascript client now solves boards with 18 or more stones locally via bare WebAssembly code running in a WebWorker. Bare WebAssembly means clang-only, without Emscripten. That in turn means no standard library, no malloc, etc. The midengine was already pretty light on memory allocation, so the main hardship was getting a stripped version of libc++ (for type_traits and similar niceties) that works in -ffreestanding mode without malloc and such. I did this by copying the libc++ headers into third_party/freestanding and adding a very large number of ifdefs. A wrinkle: the midengine is memory heavy, and WebAssembly instances can't ever shrink their overall memory size. To get around this, the client makes a fresh WebAssembly instance for each request and throws it away when compile. Thankfully the API separates compilation from instance creation, so this seems fine. Timings for an 18 stone board (https://perfect-pentago.net/#274440791932540184): MacBook Pro, 13-inch, 2019, 2.5 GHz Quad-Core Intel Core i5: Chrome 79 = 12.381 s Firefox 71 = 10.587 s Safari 13.0.4 = 12.572 s iPhone 11 Pro, iOS 13.3: 9.0 s (!) Other improvements along the way: 1. Remove the weird "boards" argument to midengine routines. All available boards are now computed inside the routines. 2. Reduce excessive templatization of midengine to help code size. This results in a small but acceptable 10-15% slowdown. 3. Make midengine work without SSE. 4. Make high_board_t smaller (16 -> 8 bytes) and leaner (no memory allocation). 5. Upgrade server to node 13.5.0. Extensive and painful binding tweaks required. 6. Use fetch instead of XMLHttpRequest in client. Posts which gave me the idea that bare WebAssembly was possible: * https://dassur.ma/things/c-to-webassembly * https://medium.com/@dougschaefer/going-straight-to-clang-for-webassembly-928df1484430

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha f1ab88d319ba84e5655f50f51c606b67fdbf2462

Solve midgames and on locally via bare WebAssembly The pentago javascript client now solves boards with 18 or more stones locally via bare WebAssembly code running in a WebWorker. Bare WebAssembly means clang-only, without Emscripten. That in turn means no standard library, no malloc, etc. The midengine was already pretty light on memory allocation, so the main hardship was getting a stripped version of libc++ (for type_traits and similar niceties) that works in -ffreestanding mode without malloc and such. I did this by copying the libc++ headers into third_party/freestanding and adding a very large number of ifdefs. A wrinkle: the midengine is memory heavy, and WebAssembly instances can't ever shrink their overall memory size. To get around this, the client makes a fresh WebAssembly instance for each request and throws it away when compile. Thankfully the API separates compilation from instance creation, so this seems fine. Timings for an 18 stone board (https://perfect-pentago.net/#274440791932540184): MacBook Pro, 13-inch, 2019, 2.5 GHz Quad-Core Intel Core i5: Chrome 79 = 12.381 s Firefox 71 = 10.587 s Safari 13.0.4 = 12.572 s iPhone 11 Pro, iOS 13.3: 9.0 s (!) Other improvements along the way: 1. Remove the weird "boards" argument to midengine routines. All available boards are now computed inside the routines. 2. Reduce excessive templatization of midengine to help code size. This results in a small but acceptable 10-15% slowdown. 3. Make midengine work without SSE. 4. Make high_board_t smaller (16 -> 8 bytes) and leaner (no memory allocation). 5. Upgrade server to node 13.5.0. Extensive and painful binding tweaks required. 6. Use fetch instead of XMLHttpRequest in client. Posts which gave me the idea that bare WebAssembly was possible: * https://dassur.ma/things/c-to-webassembly * https://medium.com/@dougschaefer/going-straight-to-clang-for-webassembly-928df1484430

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha 64528120a0efe88431d7b53b79716e1cb8da580e

Reduce external dependencies for emscripten build New command: emcc -o benchmark.js -O3 --llvm-lto 3 -s ALLOW_MEMORY_GROWTH=1 -std=c++1z -Wall -Werror -I. -Ibazel-genfiles benchmark.cc pentago/mid/{halfsuper,midengine}.cc pentago/high/board.cc bazel-bin/pentago/base/gen/tables.cc pentago/utility/debug.cc pentago/base/{board,superscore,symmetry}.cc && ls -l benchmark.*

view details

Geoffrey Irving

commit sha a44f9853a5f7b05e8aa3f9c5d45261f145a575b5

Reduce state of high_board_t

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha ec8dc299ee687a493e374702be92237bb561a4f3

Get midengine building with emscripten Command: emcc -o benchmark.js -O3 -s ALLOW_MEMORY_GROWTH=1 -std=c++1z -Wall -Werror -I. -Ibazel-genfiles -Ibazel-pentago/external/{boost,tinyformat,com_google_googletest/googletest/include} pentago/mid/{benchmark,halfsuper,midengine}.cc pentago/high/board.cc bazel-bin/pentago/base/gen/tables.cc pentago/utility/{debug,exceptions,log}.cc pentago/base/{board,count,superscore,symmetry}.cc Results: % time node benchmark.js time = 11.263 s real 0m12.034s user 0m13.689s sys 0m0.274s So a bit more than 1 s overhead starting from scratch, which seems fine.

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha fc08d25f69b63736966bc730fa40b9bc28039993

filter.h: Fix for non-sse

view details

Geoffrey Irving

commit sha 8e134267fdaa8d7b43504164b953aff88dd2cea4

Add -fno-stack-check to workaround Apple clang bug Workaround from https://forums.developer.apple.com/thread/121887.

view details

Geoffrey Irving

commit sha 0e1acc9bded7c985d7ed71c3836c5a2d0ca4285f

Partially revert e6b5b186124b7cd7ed7ea3095065f8c0206bd560 8e134267fdaa8d7b43504164b953aff88dd2cea4 is a better fix.

view details

Geoffrey Irving

commit sha 27be22f2b79c4233c618c1db97c9291bfaae0217

Port midengine to non-sse

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha a822280f14392ddb1b3625fc9a6a78e13069b110

learn: Disable broken tensorflow op build for now

view details

Geoffrey Irving

commit sha e6b5b186124b7cd7ed7ea3095065f8c0206bd560

Work around apparent Apple clang 11.0.0 bug with variable sized arrays

view details

push time in 2 months

push eventgirving/pentago

Geoffrey Irving

commit sha 8f50b284d6496213e71ae20ef20dcc1080cb35eb

midengine: Add a few spaces

view details

push time in 2 months

push eventgirving/poster

Geoffrey Irving

commit sha 1055f27474e169eca7f7e06fe696de3304368831

SConstruct: Format

view details

push time in 3 months

issue commenttensorflow/tensorflow

tf.gradients() gives the conjugate of what is expected

@martinwicke Yep. Presumably that will resolve the confusion. :)

whdc

comment created time in 5 months

fork girving/hugo-academic

📝 The website builder for Hugo. Build and deploy a beautiful website in minutes!

https://sourcethemes.com/academic/

fork in 5 months

issue commenttensorflow/tensorflow

tf.gradients() gives the conjugate of what is expected

@martinwicke I gave a talk once inside Google with a slide showing the proof that the gradient should be the conjugate. It's the TensorFlow documentation talk about gradients. Could you take a picture of that slide and attach it here?

whdc

comment created time in 5 months

issue commenttensorflow/tensorflow

tf.gradients() gives the conjugate of what is expected

If anyone is curious: @charmasaur suggests that it would have been "more correct" to define gradients the other way. This isn't correct: the easiest way to see this is to imagine that the inputs to a network are real, the outputs are real, and in the middle of the network is a holomorphic function. In this case, the optimizer has no idea that there's a holomorphic function involved in the middle of a computation: it sees a bunch of real stuff, and would have to do a full graph traversal to notice the issue.

A better option would be to define the gradient to be mathematically correct, and as @charmasaur has helpfully shown this is what TensorFlow does.

whdc

comment created time in 5 months

push eventopenai/pixel

Geoffrey Irving

commit sha 9674a621d6ab7fa60e76b6e75705a13d433f9f38

Fix most but not all npm warnings

view details

push time in 6 months

more