profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/anorth/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

drand/drand 348

🎲 A Distributed Randomness Beacon Daemon - Go implementation

ipfs/go-graphsync 52

Initial Implementation Of GraphSync Wire Protocol

filecoin-project/go-fil-markets 39

Shared Implementation of Storage and Retrieval Markets for Filecoin Node Implementations

filecoin-project/chain-validation 9

(DEPRECATED) See https://github.com/filecoin-project/test-vectors instead. (was: chain validation tools)

anorth/go-dar 4

Indexed content-addressed archive file format

ZenGround0/ent 4

Herd filecoin state trees for upgrade migrations

anorth/expl 3

An expression language for rapid, explorable, explainable programming [WIP]

ngalin/Illumimateys 2

Interview Prep.

anorth/gflows 1

Git/GitHub workflow scripts for rapid and parallel team development

issue commentfilecoin-project/lotus

Optimize snapshot creation

Ah. I figured that would make verification easier. But in that case, this is pretty easy.

Stebalien

comment created time in 5 minutes

issue commentfilecoin-project/lotus

E2E Testing for FIP8 & FIP13

  • Unit tests landed in #6432
  • Integ tests landing soon from Magik
  • Tested in a devnet
Stebalien

comment created time in 7 minutes

issue closedfilecoin-project/lotus

E2E Testing for FIP8 & FIP13

FIP8 and FIP13 will allow batched pre-commits and batched prove-commits. This process needs end-to-end testing at scale (100s of sectors, test match batch sizes).

  1. We need unit/integration tests to verify pre/prove-commit batching.
    1. What happens when a single sector fails?
  2. We need to test this in a dev-net.

closed time in 8 minutes

Stebalien

push eventfilecoin-project/go-amt-ipld

frrist

commit sha ff5bb2739f134700c471c72d7f0ef642a900caa5

fix(diff): handle edge case diffing empty against non-empty - and add tests

view details

push time in 9 minutes

push eventfilecoin-project/lotus

Łukasz Magiera

commit sha 21ba7408dd0a55115e0c485a78c78a08cc775aca

sealing: Fix restartSectors race

view details

Łukasz Magiera

commit sha d8e53593880f09e814ae63811bc9a358d84ee02f

Merge pull request #6491 from filecoin-project/fix/fsm-startup-race2 sealing: Fix restartSectors race

view details

Aayush Rajasekaran

commit sha 520a0091f54a4aee2d2595dab2798de938778cba

Merge branch 'releases' into release/v1.10.0

view details

Łukasz Magiera

commit sha 81b412399ef9a5d6956267d6471850479d1b7c91

sealing: Fix restartSectors race

view details

Jennifer

commit sha 9cb7964a7798616dd7f3468e64eeb456451b58c9

Merge pull request #6494 from filecoin-project/asr/merge-releases Merge releases into v1.10 release

view details

Jennifer Wang

commit sha c1529714e63b0ea6932da7aef2a55c8f330d7395

v1.10.0-rc4 have this version base off the right head = v1.9.0 tag

view details

Łukasz Magiera

commit sha 1f914053b43cb924766d1d9f38c8287bec8a0d48

sealing: Wire up context to batchers

view details

Łukasz Magiera

commit sha 7640ae47deba94aca2b3516c208a234f9e16bbfe

docsgen

view details

Peter Rabbitson

commit sha 3acd846dcd9a5fc425fcb2dd4463b8cb34507f01

Fix logging around mineOne - A nil MiningBaseInfo is *NOT* unexpected: it happens when one is in penalty https://github.com/filecoin-project/lotus/blob/v1.9.0/chain/stmgr/utils.go#L500-L502 - Remove the log from IsRoundWinner(): all we care about is the randbase epoch

view details

Peter Rabbitson

commit sha 429419f2101210fd3dd59ef145e0eee0286c06a3

Forgotten deadcode

view details

Peter Rabbitson

commit sha 400780606804e19fc3f77f6a9bf0a55c86ff9e8e

Incorporate the 'Time delta between now...' log into the 'completed mineOne'

view details

Peter Rabbitson

commit sha ef3ef8596081a6c19fec2c390c195b1998824bdd

Add a `lateStart` indicator, differentiate on Error/Warn/Info

view details

Jennifer Wang

commit sha c198e7aa9a67d48008bf4c3d4fdb006428367687

make lint happy

view details

Aayush Rajasekaran

commit sha 36bc96a6bc1472b2493c6165592712bc3a266028

Fix docsgen

view details

Łukasz Magiera

commit sha 55d5c11b0324230e16490023983c4d9172905396

Merge pull request #6497 from filecoin-project/feat/sealing-batcher-mctx sealing: Wire up context to batchers

view details

Łukasz Magiera

commit sha 9ab9e2a970510616afdf5e64b8ed0920041a916a

Merge pull request #6499 from filecoin-project/asr/cherry-pick-log Backport Fix logging around mineOne

view details

Aayush Rajasekaran

commit sha b9f864ba9eb29ee55573d1b84d768520e65c71eb

Expand on Drand change testing

view details

Łukasz Magiera

commit sha ae26e506825e0c48d8fd57ca06baf59ab1be8ae0

Merge pull request #6500 from filecoin-project/asr/test-drand-more Expand on Drand change testing

view details

Steven Allen

commit sha 526674cefafc15d6665726cb8ef60700aba54c7c

fix: pick the correct partitions-per-post limit

view details

Aayush Rajasekaran

commit sha c41ccde73de99622c12eaabec80d4eaa51d4ef72

Merge pull request #6503 from filecoin-project/backport/max-part-per-post backport: pick the correct partitions-per-post limit

view details

push time in 10 minutes

issue commentfilecoin-project/lotus

Create miner configuration documentation for new features introduced by FIP0008 and FIP0013

Kubus has the pr ready, update the doc for fees

jennijuju

comment created time in 10 minutes

issue commentfilecoin-project/lotus

Optimize snapshot creation

This is a bit tricky because we need to write the CAR (snapshot) in-order.

@Stebalien chain snapshots are incomplete dags already, there is no requirements for them to be ordered properly.

Stebalien

comment created time in 25 minutes

issue openedfilecoin-project/lotus

Optimize snapshot creation

Currently, snapshot creation is linear. Worse, it reads every block twice: once to write the snapshot, once to walk the graph.

As the state grows, this will become an issue. We should:

  1. Avoid the double-reads. This shouldn't be too hard and I'm not entirely sure why the current logic operates this way.
  2. Parallelize. This is a bit tricky because we need to write the CAR (snapshot) in-order. To avoid blowing up memory consumption too much, we'll need a pretty fancy algorithm here.

For 2, we'll probably want to pre-fetch blocks proportional to the current depth. I.e.:

  1. Keep a running average of the expected depth of the graph.
  2. Prefetch lookahead/2**(expected_depth - depth) (or something like that.

created time in 34 minutes

push eventfilecoin-project/lotus

Raúl Kripalani

commit sha 2e9e1c2330b9dcb3eca7fdd6682aab6a8db98176

avoid double BlockMiner instantiation.

view details

push time in an hour

push eventfilecoin-project/lotus

Raúl Kripalani

commit sha 9b2efd5ace6b0c3d7bccb8ea1c93644e0096d422

try to deflake window post itests.

view details

Raúl Kripalani

commit sha 120dd149793a7baf314c78d80bf7f50994ff610a

avoid double close.

view details

push time in 2 hours

create barnchfilecoin-project/lotus

branch : feat/maxprice-agg

created branch time in 2 hours

PR opened filecoin-project/lotus

Update default fees for aggregates

Resolves #6422

+4 -4

0 comment

1 changed file

pr created time in 2 hours

push eventfilecoin-project/lotus

Cory Schwartz

commit sha accb03719af708534be2556b2456de420980b600

address some comments

view details

push time in 3 hours

push eventfilecoin-project/lotus

Raúl Kripalani

commit sha 2a58f830c08d60efecbc7ef7c6e262571b7cd8eb

fix sector_terminate_test.go flakiness.

view details

Raúl Kripalani

commit sha 71a7270c9891088c20e80af08278e57692ceea89

cleanup gateway RPC.

view details

Raúl Kripalani

commit sha 098eb6bfff4729824bea385bd9da4d2f8f10a7e3

try using bg context on constructor.

view details

Anton Evangelatov

commit sha fc7feffaad39f11324b2d0ab7b3d3103828c6aaf

Merge branch 'raulk/itests-refactor-kit' into nonsense/split-market-miner-processes

view details

push time in 3 hours

push eventfilecoin-project/lotus

Raúl Kripalani

commit sha 2a58f830c08d60efecbc7ef7c6e262571b7cd8eb

fix sector_terminate_test.go flakiness.

view details

Raúl Kripalani

commit sha 71a7270c9891088c20e80af08278e57692ceea89

cleanup gateway RPC.

view details

Raúl Kripalani

commit sha 098eb6bfff4729824bea385bd9da4d2f8f10a7e3

try using bg context on constructor.

view details

push time in 3 hours

push eventfilecoin-project/lotus

Anton Evangelatov

commit sha aa584475cc92a514e9edd5bddc8b300a3c301c61

fix paych and sdr tests

view details

push time in 4 hours

issue openedfilecoin-project/lotus

[BUG] Unable to exit sealing state "TerminateFailed"

Describe the bug After trying to apply "lotus-miner sectors terminate" command, sector is stuck in Terminating loop.

Version (run lotus version): Happens both in 1.10.0-rc6 and lotus-miner version 1.11.0-dev+mainnet+git.6e1cf6fdc

To Reproduce Steps to reproduce the behavior:

  1. Run 'lotus-miner sectors terminate --really-do-it 1276' against broken sealing task
  2. Sector remains in TerminateFailed state
  3. Attempt lotus-miner sectors remove --really-do-it 1276 - (see errors below)

Expected behavior I expect the sector to be removed from sectors list after lotus-miner restart

Logs 2021-06-22T13:56:26.028Z INFO stores stores/remote.go:338 Delete http://192.168.1.96:3456/remote/sealed/s-t01278-1276 2021-06-22T13:56:26.065Z ERROR evtsm go-statemachine@v0.0.0-20200925024713-05bd7c71fbfe/machine.go:83 Executing event planner failed: running planner for state Terminating failed: github.com/filecoin-project/lotus/extern/storage-sealing.(*Sealing).plan /home/stuart/lotus/extern/storage-sealing/fsm.go:296

  • planner for state Terminating received unexpected event sealing.SectorRemoved ({User:{}}): github.com/filecoin-project/lotus/extern/storage-sealing.planOne.func1 /home/stuart/lotus/extern/storage-sealing/fsm.go:625

Screenshots 7. 2021-06-15 15:54:27 +0000 UTC: [event;sealing.SectorSealPreCommit2Failed] {"User":{}} seal pre commit(2) failed: storage call error 0: task aborted 8. 2021-06-15 15:55:27 +0000 UTC: [event;sealing.SectorRetrySealPreCommit2] {"User":{}} 9. 2021-06-15 17:35:07 +0000 UTC: [event;sealing.SectorSealPreCommit2Failed] {"User":{}} seal pre commit(2) failed: storage call error 101: worker restarted 37. 2021-06-16 16:56:49 +0000 UTC: [event;sealing.SectorSealPreCommit1Failed] {"User":{}} ticket expired: ticket expired: seal height: 848408, head: 851873 38. 2021-06-16 16:57:49 +0000 UTC: [event;sealing.SectorRetrySealPreCommit1] {"User":{}} 39. 2021-06-16 16:57:49 +0000 UTC: [event;sealing.SectorOldTicket] {"User":{}} 40. 2021-06-16 16:57:49 +0000 UTC: [event;sealing.SectorTicket] {"User":{"TicketValue":"ESAlNc/5Rfzf4fVi1/GsR1zWnzn+o5lMvEjn0UwWeTo=","TicketEpoch":847508}} 7070. 2021-06-22 13:55:26 +0000 UTC: [event;sealing.SectorTerminate] {"User":{}} 7071. 2021-06-22 13:55:26 +0000 UTC: [event;sealing.SectorTerminateFailed] {"User":{}} checking precommit presence: sectorNumber is allocated, but PreCommit info wasn't found on chain 7072. 2021-06-22 13:56:26 +0000 UTC: [event;sealing.SectorRemove] {"User":{}} 7073. 2021-06-22 13:56:26 +0000 UTC: [event;sealing.SectorTerminate] {"User":{}} 7074. 2021-06-22 13:56:26 +0000 UTC: [event;sealing.SectorTerminate] {"User":{}} 7075. 2021-06-22 13:56:26 +0000 UTC: [event;sealing.SectorRemoved] {"User":{}}

lotus-miner sectors list |grep 1276 1276 Terminating NO NO n/a

Additional context I have restarted the miner several times after trying to use remove command.

created time in 4 hours

issue closedfilecoin-project/lotus

[Mining Issue] go-libp2p-nat : Soap request got Http 500 Internal Serve Error

Note: For security-related bugs/issues, please follow the security policy.

Please provide all the information requested here to help us troubleshoot "mining/WinningPoSt failed" issues. If the information requested is missing, you may be asked you to provide it.

Describe the problem

After the successful Lotus-miner init operation, the following warning is issued when the Lotus-miner run operation is performed:

A libp2p related warning occurs after miner is run.

image

Mining doesn't seem to be working normally.

also, Nonce values increase slightly when checked with lotus wallet list -i image

Version

The output of lotus --version. lotus version 1.9.0+mainnet+git.ada7f97ba

Setup

You miner and daemon setup, including what hardware do you use, your environment variable settings, how do you run your miner and worker, do you use GPU and etc.

Swap : 255Gi image

Storage : /filecoine : 3.6T image

CPU Info image

Env image

Config Path image

lotus config.toml(/filecoin/lotus/node/folder/config.toml)

# Default config:
[API]
# Binding address for the Lotus API
# ListenAddress = "/ip4/127.0.0.1/tcp/9200/http"
# Not used by lotus daemon
# RemoteListenAddress = "221.229.101.4:9200"
# General network timeout value
# Timeout = "30s"

#
[Backup]
#  DisableMetadataLog = false
#
[Libp2p]
#  ListenAddresses = ["/ip4/0.0.0.0/tcp/24001"]
#  AnnounceAddresses = ["/ip4/211.229.101.4/tcp/24001"]
#  NoAnnounceAddresses = []
#  ConnMgrLow = 150
#  ConnMgrHigh = 180
#  ConnMgrGrace = "20s"
#
[Pubsub]
#  Bootstrapper = false
#  RemoteTracer = "/dns4/pubsub-tracer.filecoin.io/tcp/4001/p2p/QmTd6UvR47vUidRNZ1ZKXHrAFhqTJAD27rKL9XYghEKgKX"
#
[Client]
#  UseIpfs = false
#  IpfsOnlineMode = false
#  IpfsMAddr = ""
#  IpfsUseForRetrieval = false
#  SimultaneousTransfers = 20

[Metrics]
#  Nickname = ""
#  HeadNotifs = false

[Wallet]
#  RemoteBackend = ""
#  EnableLedger = false
#  DisableLocal = false
#
[Fees]
#  #DefaultMaxFee = "0.07 FIL"

[Chainstore]
#  EnableSplitstore = false
#  [Chainstore.Splitstore]
#    HotStoreType = "badger"
#    TrackingStoreType = ""
#    MarkSetType = ""
#    EnableFullCompaction = false
#    EnableGC = false
#    Archival = false
#

lotus-miner config.toml(/filecoin/miner/storage/config.toml)

# Default config:
[API]
  ListenAddress = "/ip4/127.0.0.1/tcp/2345/http"
  RemoteListenAddress = "127.0.0.1:2345"
  Timeout = "30s"
#
[Backup]
#  DisableMetadataLog = false
#
[Libp2p]
  ListenAddresses = ["/ip4/0.0.0.0/tcp/24001"]
  AnnounceAddresses = ["/ip4/211.229.101.4/tcp/24001"]
#  NoAnnounceAddresses = []
  ConnMgrLow = 150
  ConnMgrHigh = 180
  ConnMgrGrace = "20s"
#
[Pubsub]
  Bootstrapper = false
  RemoteTracer = "/dns4/pubsub-tracer.filecoin.io/tcp/4001/p2p/QmTd6UvR47vUidRNZ1ZKXHrAFhqTJAD27rKL9XYghEKgKX"
#
[Dealmaking]
  ConsiderOnlineStorageDeals = true
  ConsiderOfflineStorageDeals = true
  ConsiderOnlineRetrievalDeals = true
  ConsiderOfflineRetrievalDeals = true
  ConsiderVerifiedStorageDeals = true
  ConsiderUnverifiedStorageDeals = true
#  PieceCidBlocklist = []
  ExpectedSealDuration = "24h0m0s"
  PublishMsgPeriod = "1h0m0s"
  MaxDealsPerPublishMsg = 8
  MaxProviderCollateralMultiplier = 2
#  Filter = ""
#  RetrievalFilter = ""
#
[Sealing]
  MaxWaitDealsSectors = 2
  MaxSealingSectors = 0
  MaxSealingSectorsForDeals = 0
  WaitDealsDelay = "6h0m0s"
  AlwaysKeepUnsealedCopy = true
#
[Storage]
  ParallelFetchLimit = 10
  AllowAddPiece = true
  AllowPreCommit1 = true
  AllowPreCommit2 = true
  AllowCommit = true
  AllowUnseal = true
#
[Fees]
#  MaxPreCommitGasFee = "0.025 FIL"
#  MaxCommitGasFee = "0.05 FIL"
#  MaxTerminateGasFee = "0.5 FIL"
#  MaxWindowPoStGasFee = "5 FIL"
#  MaxPublishDealsFee = "0.05 FIL"
#  MaxMarketBalanceAddFee = "0.007 FIL"
#
[Addresses]
  PreCommitControl = ["f3sfwubigzqrrve6q4t3fpgwl26wmm5m7cftlw7crhyuc2umcqschv3kdu47jqbu44xj2l7y3c7twre3e55mga"]
  CommitControl = ["f3u2ww4xs74jahcdkddibkarfrqzg43tx5ixroiclybr2bd644vyj4mrdcoqqndjpwue6nmxdqtsfyzuyapika"]
  DisableOwnerFallback = false
  DisableWorkerFallback = false
#

If you have modified parts of lotus, please describe which areas were modified, and the scope of those modifications

closed time in 4 hours

KimJongYoon

issue commentfilecoin-project/lotus

[Mining Issue] go-libp2p-nat : Soap request got Http 500 Internal Serve Error

@cwhiggins This issue has been resolved since the router was replaced. Thank you.

KimJongYoon

comment created time in 4 hours

push eventfilecoin-project/lotus

Anton Evangelatov

commit sha 218c9199f3303a22bcb2133288c3053115f4f480

fix testNonGenesisMiner

view details

push time in 5 hours

push eventfilecoin-project/lotus

Anton Evangelatov

commit sha 4f2d8b0a856db53a908312c8fcded8920fd902bf

add all subsystems to deadlines and wdpost_dispute tests

view details

push time in 5 hours

issue commentfilecoin-project/lotus

[Feature Request] Prebuilt binaries for lotus in brew

Probably depends on https://github.com/filecoin-project/filecoin-ffi/pull/179

atopal

comment created time in 5 hours

issue openedfilecoin-project/lotus

[Feature Request] Prebuilt binaries for lotus in brew

Is your feature request related to a problem? Please describe. Now that lotus is in brew, the installation process for users who want to get started on MacOS is much simpler. Unfortunately brew still needs to built Lotus from scratch, which can take a significant amount of time.

Describe the solution you'd like Please offer prebuilt binaries for package managers like brew.

Describe alternatives you've considered The alternative is currently building from scratch. It works, but doesn't compare favorably with the getting started experience of other solutions in this space.

created time in 5 hours

push eventfilecoin-project/lotus

Anton Evangelatov

commit sha a828b15fbead16a30d4f0329ad526602d525ab34

revert MockSectorMgr

view details

push time in 5 hours

push eventprotocol/web3-dev-team

jnthnvctr

commit sha 629fca18f819ab59f2c2dbe2a40af3445a1befcf

Update fil-storage-homepage.md

view details

push time in 5 hours

push eventprotocol/web3-dev-team

jnthnvctr

commit sha 84d1a3516b8347dd51c4e89db72b833fa1ce846a

Rename fil-storage-homepage to fil-storage-homepage.md

view details

push time in 5 hours

push eventprotocol/web3-dev-team

jnthnvctr

commit sha cfa9e73492e66058daf3dc0323d2cb073a01f4ff

Create fil-storage-homepage

view details

push time in 5 hours

create barnchprotocol/web3-dev-team

branch : jvictor/fil-storage-homepage

created branch time in 6 hours

Pull request review commentfilecoin-project/lotus

revamped integration test kit (aka. Operation Sparks Joy)

+package kit++import (+	"bytes"+	"context"+	"crypto/rand"+	"io/ioutil"+	"sync"+	"testing"+	"time"++	"github.com/filecoin-project/go-address"+	"github.com/filecoin-project/go-state-types/abi"+	"github.com/filecoin-project/go-state-types/big"+	"github.com/filecoin-project/go-state-types/exitcode"+	"github.com/filecoin-project/go-state-types/network"+	"github.com/filecoin-project/go-storedcounter"+	"github.com/filecoin-project/lotus/api"+	"github.com/filecoin-project/lotus/api/v1api"+	"github.com/filecoin-project/lotus/build"+	"github.com/filecoin-project/lotus/chain"+	"github.com/filecoin-project/lotus/chain/actors"+	"github.com/filecoin-project/lotus/chain/actors/builtin/miner"+	"github.com/filecoin-project/lotus/chain/actors/builtin/power"+	"github.com/filecoin-project/lotus/chain/gen"+	genesis2 "github.com/filecoin-project/lotus/chain/gen/genesis"+	"github.com/filecoin-project/lotus/chain/messagepool"+	"github.com/filecoin-project/lotus/chain/types"+	"github.com/filecoin-project/lotus/chain/wallet"+	"github.com/filecoin-project/lotus/cmd/lotus-seed/seed"+	sectorstorage "github.com/filecoin-project/lotus/extern/sector-storage"+	"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"+	"github.com/filecoin-project/lotus/extern/sector-storage/mock"+	"github.com/filecoin-project/lotus/genesis"+	lotusminer "github.com/filecoin-project/lotus/miner"+	"github.com/filecoin-project/lotus/node"+	"github.com/filecoin-project/lotus/node/config"+	"github.com/filecoin-project/lotus/node/modules"+	"github.com/filecoin-project/lotus/node/modules/dtypes"+	testing2 "github.com/filecoin-project/lotus/node/modules/testing"+	"github.com/filecoin-project/lotus/node/repo"+	"github.com/filecoin-project/lotus/storage/mockstorage"+	miner2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/miner"+	power2 "github.com/filecoin-project/specs-actors/v2/actors/builtin/power"+	"github.com/ipfs/go-datastore"+	libp2pcrypto "github.com/libp2p/go-libp2p-core/crypto"+	"github.com/libp2p/go-libp2p-core/peer"+	mocknet "github.com/libp2p/go-libp2p/p2p/net/mock"+	"github.com/stretchr/testify/require"+)++func init() {+	chain.BootstrapPeerThreshold = 1+	messagepool.HeadChangeCoalesceMinDelay = time.Microsecond+	messagepool.HeadChangeCoalesceMaxDelay = 2 * time.Microsecond+	messagepool.HeadChangeCoalesceMergeInterval = 100 * time.Nanosecond+}++// Ensemble is a collection of nodes instantiated within a test.+//+// Create a new ensemble with:+//+//   ens := kit.NewEnsemble()+//+// Create full nodes and miners:+//+//   var full TestFullNode+//   var miner TestMiner+//   ens.FullNode(&full, opts...)       // populates a full node+//   ens.Miner(&miner, &full, opts...)  // populates a miner, using the full node as its chain daemon+//+// It is possible to pass functional options to set initial balances,+// presealed sectors, owner keys, etc.+//+// After the initial nodes are added, call `ens.Start()` to forge genesis+// and start the network. Mining will NOT be started automatically. It needs+// to be started explicitly by calling `BeginMining`.+//+// Nodes also need to be connected with one another, either via `ens.Connect()`+// or `ens.InterconnectAll()`. A common inchantation for simple tests is to do:+//+//   ens.InterconnectAll().BeginMining(blocktime)+//+// You can continue to add more nodes, but you must always follow with+// `ens.Start()` to activate the new nodes.+//+// The API is chainable, so it's possible to do a lot in a very succinct way:+//+//   kit.NewEnsemble().FullNode(&full).Miner(&miner, &full).Start().InterconnectAll().BeginMining()+//+// You can also find convenient fullnode:miner presets, such as 1:1, 1:2,+// and 2:1, e.g.:+//+//   kit.EnsembleMinimal()+//   kit.EnsembleOneTwo()+//   kit.EnsembleTwoOne()+//+type Ensemble struct {+	t            *testing.T+	bootstrapped bool+	genesisBlock bytes.Buffer+	mn           mocknet.Mocknet+	options      *ensembleOpts++	inactive struct {+		fullnodes []*TestFullNode+		miners    []*TestMiner+	}+	active struct {+		fullnodes []*TestFullNode+		miners    []*TestMiner+	}+	genesis struct {+		miners   []genesis.Miner+		accounts []genesis.Actor+	}+}++// NewEnsemble instantiates a new blank Ensemble.+func NewEnsemble(t *testing.T, opts ...EnsembleOpt) *Ensemble {+	options := DefaultEnsembleOpts+	for _, o := range opts {+		err := o(&options)+		require.NoError(t, err)+	}++	n := &Ensemble{t: t, options: &options}++	// add accounts from ensemble options to genesis.+	for _, acc := range options.accounts {+		n.genesis.accounts = append(n.genesis.accounts, genesis.Actor{+			Type:    genesis.TAccount,+			Balance: acc.initialBalance,+			Meta:    (&genesis.AccountMeta{Owner: acc.key.Address}).ActorMeta(),+		})+	}++	return n+}++// FullNode enrolls a new full node.+func (n *Ensemble) FullNode(full *TestFullNode, opts ...NodeOpt) *Ensemble {+	options := DefaultNodeOpts+	for _, o := range opts {+		err := o(&options)+		require.NoError(n.t, err)+	}++	key, err := wallet.GenerateKey(types.KTBLS)+	require.NoError(n.t, err)++	if !n.bootstrapped && !options.balance.IsZero() {+		// if we still haven't forged genesis, create a key+address, and assign+		// it some FIL; this will be set as the default wallet when the node is+		// started.+		genacc := genesis.Actor{+			Type:    genesis.TAccount,+			Balance: options.balance,+			Meta:    (&genesis.AccountMeta{Owner: key.Address}).ActorMeta(),+		}++		n.genesis.accounts = append(n.genesis.accounts, genacc)+	}++	*full = TestFullNode{t: n.t, options: options, DefaultKey: key}+	n.inactive.fullnodes = append(n.inactive.fullnodes, full)+	return n+}++// Miner enrolls a new miner, using the provided full node for chain+// interactions.+func (n *Ensemble) Miner(miner *TestMiner, full *TestFullNode, opts ...NodeOpt) *Ensemble {+	require.NotNil(n.t, full, "full node required when instantiating miner")++	options := DefaultNodeOpts+	for _, o := range opts {+		err := o(&options)+		require.NoError(n.t, err)+	}++	privkey, _, err := libp2pcrypto.GenerateEd25519Key(rand.Reader)+	require.NoError(n.t, err)++	peerId, err := peer.IDFromPrivateKey(privkey)+	require.NoError(n.t, err)++	tdir, err := ioutil.TempDir("", "preseal-memgen")+	require.NoError(n.t, err)++	minerCnt := len(n.inactive.miners) + len(n.active.miners)++	actorAddr, err := address.NewIDAddress(genesis2.MinerStart + uint64(minerCnt))+	require.NoError(n.t, err)++	ownerKey := options.ownerKey+	if !n.bootstrapped {+		var (+			sectors = options.sectors+			k       *types.KeyInfo+			genm    *genesis.Miner+		)++		// create the preseal commitment.+		if n.options.mockProofs {+			genm, k, err = mockstorage.PreSeal(abi.RegisteredSealProof_StackedDrg2KiBV1, actorAddr, sectors)+		} else {+			genm, k, err = seed.PreSeal(actorAddr, abi.RegisteredSealProof_StackedDrg2KiBV1, 0, sectors, tdir, []byte("make genesis mem random"), nil, true)+		}+		require.NoError(n.t, err)++		genm.PeerId = peerId++		// create an owner key, and assign it some FIL.+		ownerKey, err = wallet.NewKey(*k)+		require.NoError(n.t, err)++		genacc := genesis.Actor{+			Type:    genesis.TAccount,+			Balance: options.balance,+			Meta:    (&genesis.AccountMeta{Owner: ownerKey.Address}).ActorMeta(),+		}++		n.genesis.miners = append(n.genesis.miners, *genm)+		n.genesis.accounts = append(n.genesis.accounts, genacc)+	} else {+		require.NotNil(n.t, ownerKey, "worker key can't be null if initializing a miner after genesis")+	}++	*miner = TestMiner{+		t:          n.t,+		ActorAddr:  actorAddr,+		OwnerKey:   ownerKey,+		FullNode:   full,+		PresealDir: tdir,+		options:    options,+	}++	miner.Libp2p.PeerID = peerId+	miner.Libp2p.PrivKey = privkey++	n.inactive.miners = append(n.inactive.miners, miner)++	return n+}++// Start starts all enrolled nodes.+func (n *Ensemble) Start() *Ensemble {+	ctx, cancel := context.WithCancel(context.Background())+	n.t.Cleanup(cancel)++	var gtempl *genesis.Template+	if !n.bootstrapped {+		// We haven't been bootstrapped yet, we need to generate genesis and+		// create the networking backbone.+		gtempl = n.generateGenesis()+		n.mn = mocknet.New(ctx)+	}++	// ---------------------+	//  FULL NODES+	// ---------------------++	// Create all inactive full nodes.+	for i, full := range n.inactive.fullnodes {+		opts := []node.Option{+			node.FullAPI(&full.FullNode, node.Lite(full.options.lite)),+			node.Online(),+			node.Repo(repo.NewMemory(nil)),+			node.MockHost(n.mn),+			node.Test(),++			// so that we subscribe to pubsub topics immediately+			node.Override(new(dtypes.Bootstrapper), dtypes.Bootstrapper(true)),+		}++		// append any node builder options.+		opts = append(opts, full.options.extraNodeOpts...)++		// Either generate the genesis or inject it.+		if i == 0 && !n.bootstrapped {+			opts = append(opts, node.Override(new(modules.Genesis), testing2.MakeGenesisMem(&n.genesisBlock, *gtempl)))+		} else {+			opts = append(opts, node.Override(new(modules.Genesis), modules.LoadGenesis(n.genesisBlock.Bytes())))+		}++		// Are we mocking proofs?+		if n.options.mockProofs {+			opts = append(opts,+				node.Override(new(ffiwrapper.Verifier), mock.MockVerifier),+				node.Override(new(ffiwrapper.Prover), mock.MockProver),+			)+		}++		// Call option builders, passing active nodes as the parameter+		for _, bopt := range full.options.optBuilders {+			opts = append(opts, bopt(n.active.fullnodes))+		}++		// Construct the full node.+		stop, err := node.New(ctx, opts...)++		require.NoError(n.t, err)++		addr, err := full.WalletImport(context.Background(), &full.DefaultKey.KeyInfo)+		require.NoError(n.t, err)++		err = full.WalletSetDefault(context.Background(), addr)+		require.NoError(n.t, err)++		// Are we hitting this node through its RPC?+		if full.options.rpc {+			withRPC := fullRpc(n.t, full)+			n.inactive.fullnodes[i] = withRPC+		}++		n.t.Cleanup(func() { _ = stop(context.Background()) })++		n.active.fullnodes = append(n.active.fullnodes, full)+	}++	// If we are here, we have processed all inactive fullnodes and moved them+	// to active, so clear the slice.+	n.inactive.fullnodes = n.inactive.fullnodes[:0]++	// Link all the nodes.+	err := n.mn.LinkAll()+	require.NoError(n.t, err)++	// ---------------------+	//  MINERS+	// ---------------------++	// Create all inactive miners.+	for i, m := range n.inactive.miners {+		if n.bootstrapped {+			// this is a miner created after genesis, so it won't have a preseal.+			// we need to create it on chain.+			params, aerr := actors.SerializeParams(&power2.CreateMinerParams{+				Owner:         m.OwnerKey.Address,+				Worker:        m.OwnerKey.Address,+				SealProofType: m.options.proofType,+				Peer:          abi.PeerID(m.Libp2p.PeerID),+			})+			require.NoError(n.t, aerr)++			createStorageMinerMsg := &types.Message{+				From:  m.OwnerKey.Address,+				To:    power.Address,+				Value: big.Zero(),++				Method: power.Methods.CreateMiner,+				Params: params,++				GasLimit:   0,+				GasPremium: big.NewInt(5252),+			}+			signed, err := m.FullNode.FullNode.MpoolPushMessage(ctx, createStorageMinerMsg, nil)+			require.NoError(n.t, err)++			mw, err := m.FullNode.FullNode.StateWaitMsg(ctx, signed.Cid(), build.MessageConfidence, api.LookbackNoLimit, true)+			require.NoError(n.t, err)+			require.Equal(n.t, exitcode.Ok, mw.Receipt.ExitCode)++			var retval power2.CreateMinerReturn+			err = retval.UnmarshalCBOR(bytes.NewReader(mw.Receipt.Return))+			require.NoError(n.t, err, "failed to create miner")++			m.ActorAddr = retval.IDAddress+		}++		has, err := m.FullNode.WalletHas(ctx, m.OwnerKey.Address)+		require.NoError(n.t, err)++		// Only import the owner's full key into our companion full node, if we+		// don't have it still.+		if !has {+			_, err = m.FullNode.WalletImport(ctx, &m.OwnerKey.KeyInfo)+			require.NoError(n.t, err)+		}++		// // Set it as the default address.+		// err = m.FullNode.WalletSetDefault(ctx, m.OwnerAddr.Address)+		// require.NoError(n.t, err)++		r := repo.NewMemory(nil)++		lr, err := r.Lock(repo.StorageMiner)+		require.NoError(n.t, err)++		ks, err := lr.KeyStore()+		require.NoError(n.t, err)++		pk, err := m.Libp2p.PrivKey.Bytes()+		require.NoError(n.t, err)++		err = ks.Put("libp2p-host", types.KeyInfo{+			Type:       "libp2p-host",+			PrivateKey: pk,+		})+		require.NoError(n.t, err)++		ds, err := lr.Datastore(context.TODO(), "/metadata")+		require.NoError(n.t, err)++		err = ds.Put(datastore.NewKey("miner-address"), m.ActorAddr.Bytes())+		require.NoError(n.t, err)++		nic := storedcounter.New(ds, datastore.NewKey(modules.StorageCounterDSPrefix))+		for i := 0; i < m.options.sectors; i++ {+			_, err := nic.Next()+			require.NoError(n.t, err)+		}+		_, err = nic.Next()+		require.NoError(n.t, err)++		err = lr.Close()+		require.NoError(n.t, err)++		enc, err := actors.SerializeParams(&miner2.ChangePeerIDParams{NewID: abi.PeerID(m.Libp2p.PeerID)})+		require.NoError(n.t, err)++		msg := &types.Message{+			From:   m.OwnerKey.Address,+			To:     m.ActorAddr,+			Method: miner.Methods.ChangePeerID,+			Params: enc,+			Value:  types.NewInt(0),+		}++		_, err = m.FullNode.MpoolPushMessage(ctx, msg, nil)+		require.NoError(n.t, err)++		var mineBlock = make(chan lotusminer.MineReq)+		opts := []node.Option{+			node.StorageMiner(&m.StorageMiner),+			node.Online(),+			node.Repo(r),+			node.Test(),++			node.MockHost(n.mn),++			node.Override(new(v1api.FullNode), m.FullNode.FullNode),+			node.Override(new(*lotusminer.Miner), lotusminer.NewTestMiner(mineBlock, m.ActorAddr)),++			// disable resource filtering so that local worker gets assigned tasks+			// regardless of system pressure.+			node.Override(new(sectorstorage.SealerConfig), func() sectorstorage.SealerConfig {+				scfg := config.DefaultStorageMiner()+				scfg.Storage.ResourceFiltering = sectorstorage.ResourceFilteringDisabled+				return scfg.Storage+			}),+		}++		// append any node builder options.+		opts = append(opts, m.options.extraNodeOpts...)++		idAddr, err := address.IDFromAddress(m.ActorAddr)+		require.NoError(n.t, err)++		// preload preseals if the network still hasn't bootstrapped.+		var presealSectors []abi.SectorID+		if !n.bootstrapped {+			sectors := n.genesis.miners[i].Sectors+			for _, sector := range sectors {+				presealSectors = append(presealSectors, abi.SectorID{

It is used here, it wasn't being used on my PR, since I am not using the MockSectorMgr.

raulk

comment created time in 6 hours