profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/gaul/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Andrew Gaul gaul @Google Tokyo http://gaul.org/ Interested in entrepreneurship, distributed computing, storage, and free software. Previously at @bouncestorage, @maginatics, and @riverbed.

gaul/awesome-ld-preload 730

List of resources related to LD_PRELOAD, a mechanism for changing application behavior at run-time

gaul/are-we-consistent-yet 161

Compare eventual consistency of object stores

barnacs/compy 157

HTTP/HTTPS compression proxy

gaul/bay-area-transit-plans 19

San Francisco Bay Area Transit Plans

gaul/ActionBarSherlock 0

Library for implementing the action bar design pattern using the native action bar on Android 4.0+ and a custom implementation on pre-4.0 through a single API and theme.

gaul/aiodocker 0

Python Docker API client based on asyncio and aiohttp

gaul/amazon-glacier-cmd-interface 0

Command line interface for Amazon Glacier

gaul/amppackager 0

Tool to improve AMP URLs via Signed Exchanges

gaul/anaconda 0

A Go client library for the Twitter 1.1 API

gaul/android-components 0

A collection of Android libraries to build browsers or browser-like applications.

issue commentritazh/s3proxydocker

there some error when i run "make build" command

This project needs to upgrade from JDK 8 to JDK 11. Is there any reason you use this project instead of the upstream one? It also has a Dockerfile and image on Docker Hub.

pppaulpeter

comment created time in 13 hours

pull request comments3fs-fuse/s3fs-fuse

Change S3Proxy from memory to filesystem storage

Seems like the GitHub-supported way to due this is currently blocked on actions/virtual-environments#4085.

gaul

comment created time in 2 days

issue comments3fs-fuse/s3fs-fuse

AWS4-HMAC-SHA256

@pgodschalk You need to remove -o sigv2. eu-central-1 only supports the default sigv 4.

vivekatlantis2

comment created time in 3 days

pull request comments3fs-fuse/s3fs-fuse

Change S3Proxy from memory to filesystem storage

Homebrew recently updated to openjdk 17:

https://formulae.brew.sh/formula/openjdk#default

But it doesn't seem like this has propagated yet since Travis fails.

gaul

comment created time in 4 days

push eventgaul/s3fs-fuse

Andrew Gaul

commit sha 56ed505cc45f0e35f14eb18e37ec3c6e5b30e322

Change S3Proxy from memory to filesystem storage This should avoid OutOfMemoryErrors seen with larger object sizes. Enabled by fixes in S3Proxy 1.8.0. Also change LC_ALL to en_US.UTF-8 and add locales packages so that S3Proxy can create Unicode file names. This commit requires Java 17 on macOS to allow creating extended attributes for directories.

view details

push time in 4 days

push eventgaul/s3fs-fuse

Takeshi Nakatani

commit sha cc022a68f407cf02b417d4bd7221f2109b3ce6cb

Fixed a bug in s3fs_write

view details

Carsten Grohmann

commit sha c4ac923b4cbcf704fdf3af883cca006a88ed2e86

Ensure NUL-terminated result after strncpy (#1694) Long symlinks may cause that the result buffer is filled and not proper terminated with a null byte.

view details

Takeshi Nakatani

commit sha 5b6684ca19ff5889d06bd473c40a7da0285dab00

Fixed a bug utimens is calling before flush

view details

Takeshi Nakatani

commit sha 0555c4216eb5e64354341caa8f5dfd0c40fc0241

Added test case for utimens is calling during multipart upload

view details

Andrew Gaul

commit sha 20281737b136211291216ee84a5050dd64cd87a7

Set explicit Content-Length: 0 when initiating MPU (#1696) This works around a GCS quirk that incorrectly requires this header. Fixes #1661.

view details

Andrew Gaul

commit sha 944c1fd1483751a21a2c0400b6132654f420af2a

Clean up litter in ut_test (#1697)

view details

Takeshi Nakatani

commit sha 6edf3d6427442e185bbd888e896711ffc7b6a04a

Updated to clearly output message about the file descriptor type(pseudo/physical)

view details

Jan Stastny

commit sha a5c1915772b06e246bbb3c3cc837de02a71d669a

Set CURLOPT_UNRESTRICTED_AUTH when authenticating (#1681) This is necessary for authentication to work with AWS when it responds with 307 to the check bucket request when mounting. This happens to newly created buckets according to https://aws.amazon.com/premiumsupport/knowledge-center/s3-http-307-response/. Prior to this, curl would follow the redirect, but would not include the `Authorization` header which would end up with `404`. Co-authored-by: Takeshi Nakatani <ggtakec@gmail.com>

view details

Takeshi Nakatani

commit sha f9e80f995d12a54cf9c8b323c532670e1d4c5842

Fixed a bug about rename existing file of different sizes by mpcopy

view details

Andrew Gaul

commit sha 2154e898bcb27294a594322efc972cbef5fde50c

Fix typos (#1700)

view details

Andrew Gaul

commit sha f505c8224eccce5fe1aea00c7dd4661e057379b6

Explicitly specify lock type (#1701) This makes it more clear and type-safe if the caller already has the lock. Follows on to 84174c560d7542436067dfbfe1f697368ad7d4a1.

view details

Andrew Gaul

commit sha 2f59cb5a0a9ccb4163dd5dc6fd83a2ee26a699c0

Remove unneeded volatile qualifiers (#1702) We should use proper locking instead. Specifically for is_meta_pending this does not do what was intended due to the read-modify-write of the member.

view details

Andrew Gaul

commit sha 8948eded09500f9c9b7a12781e69ba5e98e52faf

Address clang-tidy warnings (#1703)

view details

Andrew Gaul

commit sha 1965916f7a97dd2bf77d55d7237811b28fe82c55

Convert storage class values to uppercase (#1704) AWS requires uppercase values. Fixes a regression from bbcccd6e9816ebb16ca33d2003d563b24625a053. References #1613.

view details

Andrew Gaul

commit sha 487df2700842c0a74e6314b898d726ff57a55589

Return EREMOTE when reading Glacier objects (#1705) Previously s3fs returned EIO which was not helpful to the caller. Returning a more specific error code allows automatically restoring the object via RestoreObject in a subsequent commit. References #1466.

view details

Andrew Gaul

commit sha fe44355d25dc4480b2f7fd7acd8f16c3937d1c6e

Report s3fs space as 18.4 EB (#1709) 256 TB probably seemed large in 2007 but in 2021 it is small enough to confuse some users.

view details

Andrew Gaul

commit sha 9cfa177af003d8c457eac49a3d6b9b3d290ee345

Add flag to run all tests (#1710) This allows CI to run the full set of tests while making local development faster.

view details

Takeshi Nakatani

commit sha 2d5316a3345e165b8578ae60f8941679674c6941

Fixed 1710 - Added a flag for check in Linux

view details

Takeshi Nakatani

commit sha 4cc210c5ab3397724b97307bef4af4863978266d

Prevents the etag string buffer from being destroyed during use and absorbed MPPART_INFO structure into filepart structure

view details

Martynov Maxim

commit sha 881025cc9e099a2824085133c6285d6793c09c92

Add initiator pid, uid and gid to a debug log (#1716)

view details

push time in 4 days

PR opened apache/jclouds

Removed unused jetty-server dependency
+0 -5

0 comment

1 changed file

pr created time in 5 days

create barnchgaul/jclouds-1

branch : deps/jetty-server

created branch time in 5 days

pull request commentjacobsa/fuse

Calculate pageSize and bufSize at run-time

This project is not goofys. We cannot support their issues here.

gaul

comment created time in 5 days

pull request commentjacobsa/fuse

Calculate pageSize and bufSize at run-time

https://github.com/kahing/goofys/issues

gaul

comment created time in 6 days

delete branch gaul/fuse

delete branch : page-size

delete time in 6 days

pull request commentjacobsa/fuse

Calculate pageSize and bufSize at run-time

Please use the goofys bug tracker for support requests with that software.

gaul

comment created time in 6 days

pull request commentgaul/s3proxy

Allow differentiating blobstores by buckets

@berlinsaint @jiffysrc there seems to be some overlap between #372, #374, and this PR. Can you guys sort out which approach is the best?

timuralp

comment created time in 6 days

pull request commentjacobsa/fuse

Calculate pageSize and bufSize at run-time

@gaul AWS r6g.2xlarge is arm system type,you can create a testing instance to test

I believe that Linux on ARM uses 4 KiB pages. Only macOS on ARM uses 16 KiB pages.

gaul

comment created time in 6 days

Pull request review commentgaul/s3proxy

Allow differentiating blobstores by buckets

 public static void main(String[] args) throws Exception {              String s3ProxyAuthorizationString = properties.getProperty(                     S3ProxyConstants.PROPERTY_AUTHORIZATION);+            ImmutableList.Builder<String> locatorBuckets =+                    new ImmutableList.Builder<>();+            for (String key : properties.stringPropertyNames()) {+                if (key.startsWith(S3ProxyConstants.PROPERTY_BUCKET_LOCATOR)) {+                    locatorBuckets.add(properties.getProperty(key));+                }+            }+            ImmutableList<String> buckets = locatorBuckets.build();+            String localIdentity = null;+            String localCredential = null;             if (AuthenticationType.fromString(s3ProxyAuthorizationString) !=                     AuthenticationType.NONE) {-                String localIdentity = properties.getProperty(+                localIdentity = properties.getProperty(                         S3ProxyConstants.PROPERTY_IDENTITY);-                String localCredential = properties.getProperty(+                localCredential = properties.getProperty(                         S3ProxyConstants.PROPERTY_CREDENTIAL);-                locators.put(localIdentity, Maps.immutableEntry(-                        localCredential, blobStore));+                Map.Entry<String, String> key = Maps.immutableEntry(+                        localIdentity, null);+                if (!parsedLocations.contains(key)) {

You can use the return value from parsedLocations.add.

timuralp

comment created time in 9 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgaul/s3proxy

Allow differentiating blobstores by buckets

             "s3proxy.max-single-part-object-size";     public static final String PROPERTY_V4_MAX_NON_CHUNKED_REQUEST_SIZE =             "s3proxy.v4-max-non-chunked-request-size";+    /** Used to locate blobstores by specified bucket names. */+    public static final String PROPERTY_BUCKET_LOCATOR =+            "s3proxy.bucket-locator";

I am unclear how to use this feature -- can you add some more documentation? Does each properties file have a unique identifier? Is it used for something?

timuralp

comment created time in 10 days

issue commentgaul/modernizer-maven-plugin

Java 11 Support

Yes. Do you have a specific compatibility problem?

milindbangar79

comment created time in 9 days

pull request commentGoogleCloudPlatform/gcsfuse

Upgrade jacobsa/fuse

You can fetch the pull request instead of my branch to avoid the path problems:

git fetch origin pull/552/head:your-branch-name
gaul

comment created time in 10 days

delete branch gaul/jclouds-site

delete branch : snapshot

delete time in 11 days

push eventgaul/s3proxy

Timur Alperovich

commit sha c055dfda9f9e784f09e1c4ced097b1ac1ebb27ec

README entry for the Alias middleware

view details

push time in 11 days

pull request commentapache/jclouds-site

Remove references to SNAPSHOT builds

Context:

Sorry, but the download page includes links to daily builds. As the page itself points out, these are not official ASF releases.

As such, they must not be advertised on public download pages. Such links should only be included on links intended for developers, not the general public.

https://www.apache.org/dev/release-distribution#unreleased

Please fix the download page(s) and resubmit the announce email when this has been done.

Thanks Sebb P.S. This was reported previously

gaul

comment created time in 11 days

PR opened apache/jclouds-site

Reviewers
Remove references to SNAPSHOT builds

jclouds is out of compliance with ASF policies by advertising SNAPSHOT builds to users.

+0 -65

0 comment

1 changed file

pr created time in 11 days

create barnchgaul/jclouds-site

branch : snapshot

created branch time in 11 days

pull request commentgaul/s3proxy

Alias middleware to remap backend buckets

Thank you for your contribution @timuralp! Please add a wiki page and a blurb to the README.

timuralp

comment created time in 12 days

push eventgaul/s3proxy

Timur Alperovich

commit sha 35fea54d63e0752c1df0aefba3c1363084c78926

Alias middleware to remap backend buckets Alias middleware implements a way to remap backend buckets to a configurable front-end name. The mappings are configured as: s3proxy.alias-blobstore.<s3proxy bucket> = <backend bucket> A single bucket cannot be mapped to multiple names.

view details

push time in 12 days

PR merged gaul/s3proxy

Alias middleware to remap backend buckets

Alias middleware implements a way to remap backend buckets to a configurable front-end name. The mappings are configured as: s3proxy.alias-blobstore.<s3proxy bucket> = <backend bucket>

A single bucket cannot be mapped to multiple names.

+487 -0

0 comment

4 changed files

timuralp

pr closed time in 12 days