profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/schomatis/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Lucas Molas schomatis Buenos Aires, Argentina Security Researcher

whyrusleeping/gx 1847

A package management tool

libp2p/specs 985

Technical specifications for the libp2p networking stack

whyrusleeping/gx-go 77

gx subtool for golang

whyrusleeping/cbor-gen 21

Codegen for cbor codecs on your types

issue commentipfs/go-ipfs

Automate copying of signed binaries from dist.ipfs.io/go-ipfs

@BigLep I'm not familiar with GitHub actions nor with our distribution system so I'm not sure I'm the right person for this one. However, if you think it's worthwhile I can spend some time familiarizing myself with this and give it a try.

lidel

comment created time in 6 hours

Pull request review commentipfs/go-ipfs

fix(cli): object add-link: do not allow blocks over BS limit

 Use MFS and 'files' commands instead: 			return err 		} +		// We do not allow producing blocks bigger than 1 MiB to avoid errors+		// when transmitting them over BitSwap. The 1 MiB constant is an+		// unenforced and undeclared rule of thumb hard-coded here.+		modifiedNode, err := api.Dag().Get(req.Context, p.Cid())+		if err != nil {+			return err+		}+		modifiedNodeSize, err := modifiedNode.Size()+		if err != nil {+			return err+		}+		if modifiedNodeSize > 1024 * 1024 {

@petar I see two places where we could potentially put this check deeper in the stack:

  • (api *ObjectAPI) AddLink: after the InsertNodeAtPath call (which has the actual 'insertion' logic) using a check similar to the one placed at the CLI level (fetch the modified node and check its size). We might also want to evaluate purging the modified copy from the repo.

  • (deeper) in the go-merkledag repo executing the actual insertion logic: it has the advantage that already has a permanent/temporary store division where rejecting the modified node wouldn't impact the go-ipfs repo. The downside is that we would be injecting an arbitrary restriction on a library based on the limitation of another (bitswap) just because both are consumed by the same go-ipfs logic.

Either is doable so I'll leave to you the decision of where to execute.

schomatis

comment created time in a day

PullRequestReviewEvent

push eventipfs/go-ipfs

Lucas Molas

commit sha 445b9d7d8bdf5a8baa88f32943f6136c8533a9e2

test

view details

push time in 4 days

push eventipfs/go-ipfs

Lucas Molas

commit sha 4e5e7f4b4ddcd94738850aad6d2a755ed4ecac3e

add force option

view details

Lucas Molas

commit sha e16bc392594f485a2195d8b647069337bdacbcfb

abstract check in function

view details

Lucas Molas

commit sha 6374c6d5b130250b285801f7d3249eb8821c086a

add check to all patch commands

view details

push time in 5 days

PR opened ipfs/go-ipfs

Lock MFS root in node

Draft PR intended to verify the design. Not a finished product.

The intention is to replace FilesRoot with LockedFilesRoot. To avoid a widespread code change we add the second in parallel temporarily and only lock the GC and the ipfs files write command to test the lock manually (through the added script test-mfs-lock.bash).

After design is validated FilesRoot should be replaced completely and we should look for a robust way to test the lock (preferably not through sharness/bash but through the internal Go API).

Some additional considerations:

  • We should have a read/write type of lock to avoid blocking on everything.
  • Right now we block the entire GC operation on an MFS command. We might want to decouple the MFS part of GC to reduce lock contention. (That would mainly mean fetching the MFS root's descendants separately and locking only that.)
  • This entire logic and discussion makes sense only when having a daemon running where the different GC and files commands contend for the same MFS root. If the commands are run in standalone mode then each will contend for the entire repo's lock (and this mechanism isn't relevant there).
+53 -4

0 comment

6 changed files

pr created time in 8 days

create barnchipfs/go-ipfs

branch : schomatis/review/gc-mfs-not-safe

created branch time in 8 days

delete branch ipfs/go-ipfs

delete branch : schomatis/sharness/launch-deamon-more-robust

delete time in 12 days

issue commentipfs/go-ipfs

Flaky test: TestPeersTotal

The test on which this one is based

https://github.com/ipfs/go-ipfs/blob/ef0428af407410d7f7e98ec0b3e57afd8c828031/core/corehttp/metrics_test.go#L15

has been fixed for the same error in https://github.com/libp2p/go-libp2p-swarm/commit/0538806be79cf321e79253cecaec8c9b0d2a7167.

@aschmahmann I'm not sure how to fix this as I don't know the original design rationale behind the test nor the function we are testing. The original fix changes its test to track peers instead of open connections (because of what's explained in the commit message) but in our function being tested we track connections and I imagine changing that to tracking peers instead would have unintended side effects in the callers consuming this information.

aschmahmann

comment created time in 13 days

pull request commentipfs/go-ipfs

feat(cli): add daemon option --agent-version-suffix

@lidel Thanks for pasting the diff here; I couldn't extract it from the Circle UI. Fixed the sharness test to cover both cases of the commit being present or not.

schomatis

comment created time in 13 days

push eventipfs/go-ipfs

Lucas Molas

commit sha 8662f76e5653c6ad0293635f6a044e2508148276

fix sharness test when commit is empty

view details

push time in 13 days

issue commentipfs/go-ipfs

Flaky MFS sharness test

Not much to be done at the moment so un-assigning myself to signal there is no active work being done here.

aschmahmann

comment created time in 15 days

issue commentipfs/go-ipfs

Flaky MFS sharness test

Update 9/13 (@schomatis): The summary of this flaky test (which we haven't seen failing again nor have we been able to reproduce) is that the source of the fail was ipfs files write --flush=false running locally instead of remotely through the daemon. In that scenario the flush flag is virtually ignored as we end up closing the repo after the command which will trigger a full flush from the MFS root anyway.

The reason for the command running locally instead of remotely is unknown but likely one of the following:

  • The daemon was in fact not running when it was supposed to (crashed, never started in the first place, etc.). Ideally we would need a more robust mechanism of detecting this throughout the tests (and not just at the beginning when starting the daemon) as proposed in https://github.com/ipfs/go-ipfs/issues/8311, but this is hard to implement and with low return. As a workaround to this we have added an extra check specifically for the failing test to poll the daemon's API and make sure it's running right before calling the ipfs files command (https://github.com/ipfs/go-ipfs/pull/8432). That check should be enough to uncover this potential source if it happens again (for this particular test alone).
  • The daemon was running but we executed the command locally anyway because go-ipfs-cmds failed to detect it (as explained https://github.com/ipfs/go-ipfs/issues/8311#issuecomment-889983795, it has a different detection mechanism). In that case go-ipfs-cmds silently calls the command locally and we fail to notice the difference. There is no way to detect this at the moment so https://github.com/ipfs/go-ipfs-cmds/issues/218 was created to add more granularity to the local/remote run choice and force go-ipfs-cmds to run the command remotely or fail otherwise in the specific case of the --flush=false flag being set.
aschmahmann

comment created time in 15 days

issue closedipfs/go-ipfs

Make `test_launch_ipfs_daemon` and `pollEndpoint` more reliable

Checklist

  • [X] #8312
  • [X] I am not suggesting a protocol enhancement.
  • [X] I have searched on the issue tracker for my issue.

Description

Spawned from https://github.com/ipfs/go-ipfs/issues/8131. In that test (it is likely that) the daemon was not running when it was supposed to and we caught it by chance just because there was a slight change in the behavior of a specific command that made the test fail. (Not sure if in that case the daemon failed/crashed at some point or didn't start altogether.)

We should review the pollEndpoint command and add some tests if possible. Additionally it would be useful to leave some check (pollEndpoint or other) running in the background during sharness tests and periodically check that the daemon is running.


Somewhat related to this (but should have its own issue if valuable): As an additional measure the command should be able to tell when it is intending to work through the daemon or not.

Right now we have the NoRemote/NoLocal flags at the command level (in the go-ipfs-cmds/command.go library) but we might want more granularity than that. For example in the above test the --flush=false option in ipfs files write should turn that command into a NoLocal one (to make sure we don't close and flush the local MFS root in the IPFS instance).

Maybe this requirement is too specific to this command alone and in that case just doing an independent check inside the command itself to make sure the daemon is running before executing might be enough.

closed time in 15 days

schomatis

issue commentipfs/go-ipfs

Make `test_launch_ipfs_daemon` and `pollEndpoint` more reliable

Closing this in favor of more specific actionable items like:

  • https://github.com/ipfs/go-ipfs/pull/8432
  • https://github.com/ipfs/go-ipfs-cmds/issues/218

From my review of the sharness test it seems that adding some concurrent check for the daemon to be up in parallel with the running tests in bash is a very involved task and it doesn't seem to have high value at the moment (this should be revisited if we hit more flaky tests like https://github.com/ipfs/go-ipfs/issues/8131).

schomatis

comment created time in 15 days

pull request commentipfs/go-ipfs

feat(cli): add daemon option --agent-version-suffix

Rebased. Still hitting same error.

schomatis

comment created time in 15 days

create barnchipfs/go-ipfs

branch : schomatis/flaky-mfs-test/add-extra-poll-check

created branch time in 15 days

issue openedipfs/go-ipfs-cmds

Allow NoRemote/NoLocal to be configured on a per-run basis

https://github.com/ipfs/go-ipfs-cmds/blob/87b5c50105f87fafa651000c4955571b132948b2/command.go#L94-L98

At the moment these configurations apply on a per-command basis. For some specific cases of go-ipfs commands it might be valuable to configure them on a per-run basis depending on the arguments the command is being run with.

More background (see also https://github.com/ipfs/go-ipfs/issues/8131): ipfs files write command can be run either locally or remotely but when specified with the --flush=false flag it should be run only remotely or fail to make sure the flush doesn't take place.

created time in 15 days

push eventipfs/go-ipfs

guseggert

commit sha b2c3959a4f882235068fc10088844c6fddc7e7e6

ci: use dynamic config for CircleCI When developing against ipfs/go-ipfs, we would like to be able to use the 2xlarge resource class for faster build and test cycles, but many external contributers will not have this resource class available to them. There is no direct way to change the resource class, so this uses dynamic config to generate a parameters JSON obj which is then fed into the configuration when the workflow starts, based on the Git URL of the build. For repos other than ipfs/go-ipfs, this reverts back to the "medium" resource class with a Make job parallelism of 3.

view details

guseggert

commit sha ef0428af407410d7f7e98ec0b3e57afd8c828031

ci: drop unit tests make jobs back to 1 This was accidentally bumped up, but it doesn't need to be and slows things down rather than helps.

view details

Lucas Molas

commit sha a54d54b8f29c9dc076546aab3e51cc6fef21779d

feat(cli): add daemon option --agent-version-suffix

view details

push time in 15 days

pull request commentipfs/go-ipfs

feat(cli): add daemon option --agent-version-suffix

(The tests themselves are passing, the error seems unrelated to this PR and more internal with CI dynamics but let me know if you'd like another run or any modification on this patch.)

schomatis

comment created time in 18 days

pull request commentipfs/go-ipfs

feat(cli): add daemon option --agent-version-suffix

@lidel Sharness passing now (didn't really do anything besides squashing and triggering another round of testing; this was already passing locally on my machine).

schomatis

comment created time in 18 days

push eventipfs/go-ipfs

Lucas Molas

commit sha f5ba05a1a0e416f103bd0a5352d3278eea3cf18e

feat(cli): add daemon option --agent-version-suffix

view details

push time in 18 days

create barnchipfs/go-ipfs

branch : schomatis/sharness/launch-deamon-more-robust

created branch time in 19 days

push eventipfs/go-ipfs

Lucas Molas

commit sha 4523363237dcd0bc08588fcaefa2b214480f7098

minor fixes

view details

push time in 19 days

push eventipfs/go-ipfs

Lucas Molas

commit sha 4038f6f2d9407341709f5b01f0767b129402f241

refactor: move logic to version.go

view details

push time in 19 days

push eventipfs/go-ipfs

Lucas Molas

commit sha 0fe614b764395ef370e88fba3bf604c26d156a2d

WIP sharness test

view details

Lucas Molas

commit sha ba81fdda57d25e61f68c7e422b7fa2f5153d5af8

Merge branch 'schomatis/cli/agent-version-suffix' of https://github.com/ipfs/go-ipfs into schomatis/cli/agent-version-suffix

view details

push time in 19 days

Pull request review commentipfs/go-ipfs

feat(cli): add daemon option --agent-version-suffix

 type IpfsNode struct { 	// Self 	Identity peer.ID // the local node's identity +	// FIXME: Check if this is the right place for it.+	UserAgentSuffix string

Sounds good, thanks for the confirmation.

schomatis

comment created time in 19 days

PullRequestReviewEvent

push eventipfs/go-ipfs

Lucas Molas

commit sha f0b360ed3469637e1c0f573cf90b2722173dcbdd

Update cmd/ipfs/daemon.go Co-authored-by: Marcin Rataj <lidel@lidel.org>

view details

push time in 19 days