profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/dirkmc/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

dirkmc/ace 1

Ajax.org Cloud9 Editor

dirkmc/filecoin-deal-proofs-svc 1

Publish a list of Filecoin deals with merkle proofs

dirkmc/go-ipfs 1

IPFS implementation in go

dirkmc/js-ipfs-repo 1

Implementation of the IPFS Repo spec in JavaScript

dirkmc/js-ipns 1

Utilities for creating, parsing, and validating IPNS records

dirkmc/cbor-gen 0

Codegen for cbor codecs on your types

dirkmc/debounce 0

A debouncer written in Go.

Pull request review commentfilecoin-project/go-data-transfer

fix potential deadlock on channel shutdown

 func (c *dtChannel) open( 	if c.gsKey != nil { 		// Cancel the existing graphsync request 		completed := c.completed-		err := c.cancelRequest(ctx)+		errch := c.cancelRequest(ctx)++		// Wait for the complete callback to be called+		err := waitForCompleteHook(ctx, completed) 		if err != nil {-			return nil, xerrors.Errorf("restarting graphsync request: %w", err)+			return nil, xerrors.Errorf("%s: waiting for cancelled graphsync request to complete: %w", chid, err) 		} -		// Wait for the cancel to go through-		err = waitForCancelComplete(ctx, completed)

See above for why we want to wait for complete and then wait for cancel to return (cancel now happens in a go routine)

dirkmc

comment created time in 16 hours

PullRequestReviewEvent

Pull request review commentfilecoin-project/go-data-transfer

fix potential deadlock on channel shutdown

 func (c *dtChannel) open( 	if c.gsKey != nil { 		// Cancel the existing graphsync request 		completed := c.completed-		err := c.cancelRequest(ctx)+		errch := c.cancelRequest(ctx)++		// Wait for the complete callback to be called+		err := waitForCompleteHook(ctx, completed)

Yeah this is quite a tricky interface:

  • When we call graphsync.CancelRequest it
    • drains the event queue
    • calls the complete callback
    • returns error / nil

So from the go-data-transfer layer we need to wait for the complete hook to get called, then wait for the cancel method to return.

The complete hook deliberately doesn't take the lock - it closes a channel. This means we can keep the lock while waiting for complete. If we were to release the lock while waiting, potentially another restart could interrupt the ongoing restart and that makes things very hard to reason about. So it's simpler just to keep the lock for the duration of the open() method.

dirkmc

comment created time in 16 hours

PullRequestReviewEvent

PR opened filecoin-project/boost

sketch of DB implementation
+136 -0

0 comment

6 changed files

pr created time in 16 hours

create barnchfilecoin-project/boost

branch : feat/db-sketch

created branch time in 16 hours

PullRequestReviewEvent

Pull request review commentfilecoin-project/boost

Outer layer of boost with deal validation

+package sqlstore

Suggest calling this dealstore instead of sqlstore, as it probably won't be SQL specific

aarshkshah1992

comment created time in 16 hours

Pull request review commentfilecoin-project/boost

Outer layer of boost with deal validation

+package types++// Status is the status of a boost storage deal and represents an unit of resumption.+type Status int++const (+	New Status = iota+	Transferring+	Publishing+	Sealing+	Complete+	Failed+	Cancelled+)

I suggest we keep states to the minimum required for resumption, as state management has historically been the trickiest part:

  • Transferring
  • Publishing
  • Sealing
  • Complete

I don't believe we need a New state, and for the purposes of resumption Complete / Failed / Cancelled are equivalent. We can probably distinguish between them for the UI using flags.

aarshkshah1992

comment created time in 16 hours

PullRequestReviewEvent

issue commentfilecoin-project/go-data-transfer

Deadlock in dtChannel mutex

The solution I suggested here doesn't work because when the graphsync request is created we have to wait for a callback to get the graphsync key associated with the request. So in the end I went with a smaller change to the code: https://github.com/filecoin-project/go-data-transfer/pull/278

hannahhoward

comment created time in 18 hours

PR opened filecoin-project/go-data-transfer

fix potential deadlock on channel shutdown

Fixes https://github.com/filecoin-project/go-data-transfer/issues/277

Move method calls to cancel request / response outside the channel lock, because

  • calling cancel on the channel drains the queue of events
  • when the event callbacks are called they may try to take that same lock
+91 -38

0 comment

1 changed file

pr created time in 18 hours

create barnchfilecoin-project/go-data-transfer

branch : fix/gs-dl

created branch time in 18 hours

issue commentfilecoin-project/go-data-transfer

Deadlock in dtChannel mutex

The logic is quite complex at the moment, because when a transfer is restarted we need to

  • cancel the current graphsync request for data
  • wait for the cancel to complete
  • start a new request for data

The same object is used to track channel state before and after a restart, even though the graphsync key changes. This makes the locking quite tricky.

I think the best way to solve this is

  • each time a request for data is made (new or restart)
    • create a new object to track the state of the channel
    • add the object to a map of graphsync key -> channel state object
  • when graphsync calls hooks with a graphsync key, map from the key to a channel state object
  • when a transfer is cancelled
    • create a cancelling channel on the object
    • wait for cancel to complete in a go-routine, and close the channel when it does
  • when graphsync calls a hook on a channel that has been cancelled, return immediately from the hook
hannahhoward

comment created time in 20 hours

delete branch filecoin-project/boost

delete branch : feat/ux-fixes

delete time in 4 days

push eventfilecoin-project/boost

Dirk McCormick

commit sha 094f0a918db6be36e410fe953b5aba0200f5d91b

fix: ux - rename package.json scripts

view details

dirkmc

commit sha c202a9fb0f5a3bd48f507f8adfd8c8edf73ed3e3

Merge pull request #3 from filecoin-project/feat/ux-fixes ux: rename package.json scripts

view details

push time in 4 days

PR opened filecoin-project/boost

ux: rename package.json scripts
+5 -2

0 comment

2 changed files

pr created time in 4 days

create barnchfilecoin-project/boost

branch : feat/ux-fixes

created branch time in 4 days

delete branch filecoin-project/boost

delete branch : feat/uxv1

delete time in 5 days

PR merged filecoin-project/boost

ux v0
+571 -0

0 comment

5 changed files

dirkmc

pr closed time in 5 days

push eventfilecoin-project/boost

Dirk McCormick

commit sha b9c00ccc77eee6cd8a8226c27eda7139397103a1

ux v0

view details

dirkmc

commit sha dc4354e16aaa5fcffcbd3e761192aeb9d76c9d09

Merge pull request #2 from filecoin-project/feat/uxv1 ux v0

view details

push time in 5 days

MemberEvent

PR opened filecoin-project/boost

ux v0
+571 -0

0 comment

5 changed files

pr created time in 5 days

create barnchfilecoin-project/boost

branch : feat/uxv1

created branch time in 5 days

delete branch filecoin-project/boost

delete branch : nonsense/initial-boost-service

delete time in 6 days

push eventfilecoin-project/boost

Anton Evangelatov

commit sha 7d87af31e1a61fccf03baa2a1e66e558872e0f30

initial boost service

view details

Anton Evangelatov

commit sha 3d76de30afa088a1ac320499d372763827a851e7

init boost libp2p node

view details

Anton Evangelatov

commit sha 9badceba6eeb0c1f0fd3d1675df1fa3129055037

cleanup

view details

Anton Evangelatov

commit sha 0eb323dbb2ac0625a1de4bd2a5d5ba68d470b8a1

fixup libp2p node

view details

Anton Evangelatov

commit sha 4af27857fba0100226e2c38a423ab216a7c4d7c3

remove gossip

view details

Anton Evangelatov

commit sha 29f046d76f8d4c2055e9f5f9961256e3c2bb90f1

logs

view details

Anton Evangelatov

commit sha 045fe7beb6ac5ce9e876ffa4c75660f48e43dfbd

remove further

view details

dirkmc

commit sha 03779ac13353b6118762c1853c86994cb1f42597

Merge pull request #1 from filecoin-project/nonsense/initial-boost-service initial boost service boilerplate

view details

push time in 6 days

PR merged filecoin-project/boost

Reviewers
initial boost service boilerplate

Copied over from Lotus:

  • Config
  • Logging
  • Repo
  • Dependency injection
  • Code generation for JSON RPC server/client
  • Build process / Makefile
  • Service instantiation
  • Boost libp2p node

TODO FOLLOWUP:

  • [ ] removing of any remaining code that is not markets related (node/modules/storageminer.go, node/impl, node/config, ...)
  • [ ] maybe remove dht from boost libp2p node
  • [ ] maybe remove repo datastore
+15767 -0

0 comment

96 changed files

nonsense

pr closed time in 6 days

PullRequestReviewEvent