profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/hsanjuan/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Hector Sanjuan hsanjuan @protocol Madrid

filecoin-project/filecoin-docs 108

Filecoin Docs

hsanjuan/dccpi 44

A simple, easy to use, NMRA Digital Command Control (DCC) Python implementation for Raspberry Pi

hsanjuan/ansible-ipfs-cluster 33

Ansible roles for go-ipfs and ipfs-cluster

filecoin-project/lily 25

capturing on-chain state for the filecoin network

filecoin-project/sentinel 19

Filecoin Network monitoring and analysis tools.

hsanjuan/crubyflie 9

A Ruby client for Crazyflie

ayoub-benali/signal-desktop-client 6

none official native desktop client for signal

filecoin-project/network-info 6

Information about active Filecoin networks and a place to report network issues

filecoin-project/sentinel-drone 5

An analytics capture agent for lotus daemon which is forked from Telegraf (https://github.com/influxdata/telegraf) to support custom input/output plugins.

filecoin-project/lotus-docs 4

Documentation for Lotus

push eventipfs/ipfs-cluster-website

Hector Sanjuan

commit sha 58d4349f59b097a9b8dfa1847e4afdf77c44c02b

Explain how to overwrite identity using env-vars

view details

push time in 2 days

push eventipfs/notes

Elton SV

commit sha f86b838d86119638964b13b571859f235a7a61aa

Update README.md

view details

Hector Sanjuan

commit sha c899f207e5bf29b444d6cf9c8c07e50f0d1f7a03

Merge pull request #450 from el10savio/crdts-for-fun-and-profit Added `CRDTs For Fun and Profit` To CRDT Research List

view details

push time in 3 days

PR merged ipfs/notes

Added `CRDTs For Fun and Profit` To CRDT Research List

Updated CRDT Research README.md with a link to CRDTs For Fun and Profit

+1 -0

0 comment

1 changed file

el10savio

pr closed time in 3 days

issue commentipfs/ipfs-cluster

Pin status with retries

I can offer an approach to pin retry improvements after #1484 .

The pintracker can have several tracks (implemented as pin workers):

  • Track A for pins that have been "recently" pinned
  • Track B for pins that were pinned some time ago (i.e. > 24h ago)

This would put new-pins in "front of the queue". Pins that are older and that keep failing or retrying would still be retried but would end on a different queue. A "bump" could be performed by repinning.

A cluster peer restart, would not have to go through all the pins that have failed, as old ones would naturally end up in the Track B. The cluster would be able to start pinning newer pins using the specific Track A.

This is a low-complexity solution though and keeps the "stateless pintracker" stateless and changes are going to be quite self-contained.


The alternative is to add state to the tracker so that I can track metadata (how many times things have been retried etc). Cluster peers do not store anything other than the pinset (shared state, consensus). This requires permanent storage outside "shared state", which requires creating and using a new part of the datastore (namespace).

This is going to be specially problematic with raft, as raft nodes do not use a permanent ipfs datastore (instead deserialize the state into an in-mem datastore). We could say "raft doesn't support this", but in any case things get more difficult.

Once we had state for that, the "stateless" pin tracker would use that state to decide if something is worth retrying again. But of course we will need a way to tell it that something is indeed worth retrying, that something needs to be retried "now" etc. so there is a number of change in semantics that would need to get supported, potentially also ending up in a pinning multi-track solution.


@alanshaw thoughts? Do you think the first option could actually alleviate most problems?

hsanjuan

comment created time in 3 days

pull request commentipfs/ipfs-cluster

Add: include a timestamp that tracks when a Pin was added to the state

Fixes #989 .

hsanjuan

comment created time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 24d8b59609a2580552fa97929d77745a6ef098c9

Fix config test

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha c329c3bf8edbc229e8e7a5ffa4c9b95d301fd9f9

Fix panic in common api tests

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 263c17c383a319f003245b308f226d8c5427ffd6

Changelog for v0.14.1

view details

Hector Sanjuan

commit sha 506b1657b9a3f5702fe9fb7f9f471d79173c15a0

Merge pull request #1452 from ipfs/v0.14.1/changelog Changelog for v0.14.1

view details

Hector Sanjuan

commit sha 0162df60eb28d73099fb28fc120f19d2d790138b

Release v0.14.1

view details

Hector Sanjuan

commit sha 267deeff28114814c426d79ccffcbe1f422b3da7

Set development version v0.14.1-next

view details

Hector Sanjuan

commit sha f9b20caa2a7d4005ddbadfe42e1b6e8db04c1944

changelog: fix v0.14.1 release date

view details

Hector Sanjuan

commit sha 26007114b4e971b5aeacf31fd51565a9b6fc92c8

Adder: try to make test less flaky

view details

Hector Sanjuan

commit sha 396a348a6541177a61e91bfd8c1d50ef75822559

Merge pull request #1462 from ipfs/fix/1461-flaky-test Adder: try to make test less flaky

view details

Hector Sanjuan

commit sha 3e54c4c6954684e2839c4296a5f022a91a554548

Informer/disk: deprecate RepoSize metric The allocator is hardcoded to descendalloc for freespace so it is not even useful as it would allocate to peers with largest reposize first no matter what. We are, in any case, reworking allocators and informers etc.

view details

Hector Sanjuan

commit sha 4060f4196b32d482a4d520fadbb28c824b1c2aba

Revert "Informer/disk: deprecate RepoSize metric" This reverts commit 3e54c4c6954684e2839c4296a5f022a91a554548.

view details

Hector Sanjuan

commit sha b6a46cd8a4ce12b4b39cd64468f2f2b4cfd3a4ce

allocator: rework the whole allocator system The new "metrics" allocator is about to partition metrics and distribe allocations among the partitions. For example: given a region, an availability zone and free space on disk, the allocator would be able to choose allocations by distributing among regions and availability zones as much as possible, and for those peers in the same region/az, selecting those with most free space first. This requires a major overhaul of the allocator component.

view details

Hector Sanjuan

commit sha ea5e18078caa990c06788ec4a7f892b8d1dfca53

Informers: GetMetric() -> GetMetrics() Support returning multiple metrics per informer.

view details

Hector Sanjuan

commit sha 1c0abde8a5684f9597e91f935546510c02ee46ed

Informer: add tags informer The tags informer produces metrics in the form tags:name/value with the name and values defined in its configuration.

view details

Hector Sanjuan

commit sha cf4fb74993de57f63a1a6d9f984de2b9a92ac4d3

service/follow: Enable new metrics allocator

view details

Hector Sanjuan

commit sha 63972f2b2e7930009e95cc9f1e2b1b715cae4587

API: Refactor REST API. Extract all functionality. This is a preparatory PR to add additional APIs (Pinning Service API) easily to cluster. Instead of copy-pasting most of what the REST API does, I have refactored so that the whole configuration, routing and request-handling utilities can be re-used. The worst part has been to divide the test between tests that test core (common.API) functionality and tests that test specific REST API endpoint functionality. I could not get away without an additional common/test package to provide functions that are used from both places. This is a side effect of testing both http and libp2p endpoints for every request etc.

view details

Hector Sanjuan

commit sha 0dfe1ae0630d51fea0947ccb1b2fcf745da57e78

Merge pull request #1471 from ipfs/feat/refactor-api API: Refactor REST API to enable re-use of functionality.

view details

Hector Sanjuan

commit sha 6b31f443512e6b89a70b76633d64c619cf0ab806

Address most comments from PR review

view details

Hector Sanjuan

commit sha 26e229df94085d472dfa7809f7a683fcb7304f21

Rename allocator/metrics to allocator/balanced

view details

Hector Sanjuan

commit sha db00d651bf30f39a1b8a8614db5bc4040aa638c4

Balanced allocator: weight-based ordering of partitions This fixes the issue about partitions not being picked based on the amount of freespace available in them. It additionally removes the metrics registry and carries information directly in the metric. Metrics have two additional fields: Weight and Partitionable. Informers have been updated to make use of these fields. Partitions have weights that equals to the weight of the metrics under them. Older cluster versions will not set these fields. Partitionable is false by default and weight has a GetWeight() function to convert value->weight when unset. This provides backwards compatibility for the freespace metric.

view details

Hector Sanjuan

commit sha 96db605c5027dbdbf69d93aa1b3a942d76c5811f

Merge pull request #1468 from ipfs/fix/159-improved-allocators Add tags informer and enable partition-based peer allocations for intelligent distribution

view details

Hector Sanjuan

commit sha af9313d4fec6b802547164d257beae5c841bf082

Remove test files that should have not been committed

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 066b1f45149951a6448cdb42a9332cfe0466e32b

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

Hector Sanjuan

commit sha a741e3067d8cb53f4454bac5bcb4c9451e132fa0

Merge pull request #1486 from ipfs/feat/github-actions Move testing pipeline to github actions.

view details

Hector Sanjuan

commit sha e9857652f2e30b02403464122cca49ca65832ece

Add a timestamp to Pins This adds a Timestamp field to the pin objects. This allows to track when they were pinned. This: * Allows the pin-tracker to actually show accurate information on when the pin entered the system for pins that are not part of ongoing operations (currently it shows time.Now()) * Adds support for reporting timestamp on a pinning services api.

view details

Hector Sanjuan

commit sha c7760513977796d2d510819d6f2263db3814bb66

ctl: Include "added" column in pin ls command Also, change format from humanize to "yyyy-mm-dd HH:MM:SS" for times.

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 066b1f45149951a6448cdb42a9332cfe0466e32b

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

Hector Sanjuan

commit sha a741e3067d8cb53f4454bac5bcb4c9451e132fa0

Merge pull request #1486 from ipfs/feat/github-actions Move testing pipeline to github actions.

view details

Hector Sanjuan

commit sha 1dca6e0ecae6e2cb24db03e73e0b825886fcf98e

allocator: balanced allocator name should be "balanced" It was an oversight to not rename this.

view details

Hector Sanjuan

commit sha 3f895df8cf59f4daeac46a418aa6c6a0404deb3e

Fix: backwards compatibility for allocator default settings When the allocator is not defined in the configuration, it will take defaults and assume there is a "tags" informer. That is not the case. When not defined, we assume it should allocate only by "freespace".

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 066b1f45149951a6448cdb42a9332cfe0466e32b

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

Hector Sanjuan

commit sha a741e3067d8cb53f4454bac5bcb4c9451e132fa0

Merge pull request #1486 from ipfs/feat/github-actions Move testing pipeline to github actions.

view details

Hector Sanjuan

commit sha ac1694581128d0f806971b838020735323e527a9

api: Support a custom error function for custom error messages

view details

Hector Sanjuan

commit sha c441158562a8625eba21105bd9a8b3111af3e8b8

cluster: add PeersWithFilter option that only requests info for certain peer IDs (currently will be unused)

view details

Hector Sanjuan

commit sha de22471aa3239cb9643e35acbbc110166d39b4b0

common api: automatically set NotFound errors

view details

Hector Sanjuan

commit sha 1cb9a944b2997140b4b34bf59312293cbdb58788

PinInfo type: include Allocations, Origins, Metadata This will facilitate building outputs for the Pinning Services API, saving a round trip to query the cluster State, since all the needed information already comes from the PinTracker, which has already accessed the state. Since the pintracker already included a state attribute (Name), we are simply going down that path.

view details

Hector Sanjuan

commit sha 4cbc197efb10982cd505924fe1afd1501b66504e

api: Add the Pinning Services API module This fixes #1213. It adds partial support for the Pinning Services API with some caveats: * RequestIDs == CIDs. This is a violation of the spec, as request IDs will not be unique etc. * Pagination, name matching, metadata matching, ordering etc. are not supported in the List endpoint. * The List endpoint only supports status filtering and cid query parameter. * Created time property is not supported and always set to Now() There is more work to do here: cleanup, extract useful types etc. and TESTS.

view details

Hector Sanjuan

commit sha 0147954f9ed35b44fdd55e766b891b9a6df1552c

pinsvcapi: extract types into their own module for better re-use later

view details

Hector Sanjuan

commit sha 5f5ffdcb7ef37f6158e95b8a8ab7407b652d5681

Wire in pinsvcapi to ipfs-cluster-service

view details

push time in 3 days

delete branch ipfs/ipfs-cluster

delete branch : feat/github-actions

delete time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 066b1f45149951a6448cdb42a9332cfe0466e32b

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

Hector Sanjuan

commit sha a741e3067d8cb53f4454bac5bcb4c9451e132fa0

Merge pull request #1486 from ipfs/feat/github-actions Move testing pipeline to github actions.

view details

push time in 3 days

PR merged ipfs/ipfs-cluster

Move testing pipeline to github actions.

Since travis cannot stop sucking.

+126 -53

0 comment

2 changed files

hsanjuan

pr closed time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 066b1f45149951a6448cdb42a9332cfe0466e32b

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha e6c26785f8798eebc5c0ee3e052fbb5f3d399c13

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 7ce87e3d21c560b8fad078e403550aa239d63ab0

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

push time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 41d27d61faf8d5627f10fff360260b61801f2f75

Move testing pipeline to github actions. Since travis cannot stop sucking.

view details

push time in 3 days

PR opened ipfs/ipfs-cluster

Move testing pipeline to github actions.

Since travis cannot stop sucking.

+104 -53

0 comment

2 changed files

pr created time in 3 days

create barnchipfs/ipfs-cluster

branch : feat/github-actions

created branch time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha a7512e327ce89a695301f2eee4eb4a707b5ea813

pinsvcapi: Fix assign before append.

view details

push time in 3 days

PR opened ipfs/ipfs-cluster

Fix: backwards compatibility of the new allocator

Make sure that we can still pin things as before with the new cluster version by accounting that users will have no allocator-configuration in their service.json.

+9 -2

0 comment

2 changed files

pr created time in 3 days

PR opened ipfs/ipfs-cluster

Add: include a timestamp that tracks when a Pin was added to the state

This can be used to provide actual timestamp information when checking pin status. We can see how "old" pins are, in addition to how long they have been queued etc.

+81 -40

0 comment

6 changed files

pr created time in 3 days

PR opened ipfs/ipfs-cluster

Feat: add a new API component that implements the Pinning services API

This branch adds a new API component which is very similar to the REST API component (using the common code that was extracted from it), only to implement the Pinning Services API (or the parts that we can).

WIP

+959 -65

0 comment

16 changed files

pr created time in 3 days

create barnchipfs/ipfs-cluster

branch : fix/default-allocator-compat

created branch time in 3 days

create barnchipfs/ipfs-cluster

branch : feat/pin-time

created branch time in 3 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha 3e54c4c6954684e2839c4296a5f022a91a554548

Informer/disk: deprecate RepoSize metric The allocator is hardcoded to descendalloc for freespace so it is not even useful as it would allocate to peers with largest reposize first no matter what. We are, in any case, reworking allocators and informers etc.

view details

Hector Sanjuan

commit sha 4060f4196b32d482a4d520fadbb28c824b1c2aba

Revert "Informer/disk: deprecate RepoSize metric" This reverts commit 3e54c4c6954684e2839c4296a5f022a91a554548.

view details

Hector Sanjuan

commit sha b6a46cd8a4ce12b4b39cd64468f2f2b4cfd3a4ce

allocator: rework the whole allocator system The new "metrics" allocator is about to partition metrics and distribe allocations among the partitions. For example: given a region, an availability zone and free space on disk, the allocator would be able to choose allocations by distributing among regions and availability zones as much as possible, and for those peers in the same region/az, selecting those with most free space first. This requires a major overhaul of the allocator component.

view details

Hector Sanjuan

commit sha ea5e18078caa990c06788ec4a7f892b8d1dfca53

Informers: GetMetric() -> GetMetrics() Support returning multiple metrics per informer.

view details

Hector Sanjuan

commit sha 1c0abde8a5684f9597e91f935546510c02ee46ed

Informer: add tags informer The tags informer produces metrics in the form tags:name/value with the name and values defined in its configuration.

view details

Hector Sanjuan

commit sha cf4fb74993de57f63a1a6d9f984de2b9a92ac4d3

service/follow: Enable new metrics allocator

view details

Hector Sanjuan

commit sha 6b31f443512e6b89a70b76633d64c619cf0ab806

Address most comments from PR review

view details

Hector Sanjuan

commit sha 26e229df94085d472dfa7809f7a683fcb7304f21

Rename allocator/metrics to allocator/balanced

view details

Hector Sanjuan

commit sha db00d651bf30f39a1b8a8614db5bc4040aa638c4

Balanced allocator: weight-based ordering of partitions This fixes the issue about partitions not being picked based on the amount of freespace available in them. It additionally removes the metrics registry and carries information directly in the metric. Metrics have two additional fields: Weight and Partitionable. Informers have been updated to make use of these fields. Partitions have weights that equals to the weight of the metrics under them. Older cluster versions will not set these fields. Partitionable is false by default and weight has a GetWeight() function to convert value->weight when unset. This provides backwards compatibility for the freespace metric.

view details

Hector Sanjuan

commit sha 96db605c5027dbdbf69d93aa1b3a942d76c5811f

Merge pull request #1468 from ipfs/fix/159-improved-allocators Add tags informer and enable partition-based peer allocations for intelligent distribution

view details

Hector Sanjuan

commit sha af9313d4fec6b802547164d257beae5c841bf082

Remove test files that should have not been committed

view details

Hector Sanjuan

commit sha 77d53cc6f1793c852136744bbdd402886edd9576

api: Support a custom error function for custom error messages

view details

Hector Sanjuan

commit sha 4f70b44874a26a59afaddb9fa4b39ca7ac585162

cluster: add PeersWithFilter option that only requests info for certain peer IDs (currently will be unused)

view details

Hector Sanjuan

commit sha bcefc52a132dc90bca54ee5ac4825030b853ebd0

common api: automatically set NotFound errors

view details

Hector Sanjuan

commit sha a828afcb36525b5b21ed1e8ac602e57e26d8f2ff

PinInfo type: include Allocations, Origins, Metadata This will facilitate building outputs for the Pinning Services API, saving a round trip to query the cluster State, since all the needed information already comes from the PinTracker, which has already accessed the state. Since the pintracker already included a state attribute (Name), we are simply going down that path.

view details

Hector Sanjuan

commit sha efd63c9367abf15d677a1e94750ebd740c2a0115

api: Add the Pinning Services API module This fixes #1213. It adds partial support for the Pinning Services API with some caveats: * RequestIDs == CIDs. This is a violation of the spec, as request IDs will not be unique etc. * Pagination, name matching, metadata matching, ordering etc. are not supported in the List endpoint. * The List endpoint only supports status filtering and cid query parameter. * Created time property is not supported and always set to Now() There is more work to do here: cleanup, extract useful types etc. and TESTS.

view details

Hector Sanjuan

commit sha abf247fcc33de0f4ecf3574202c4b66de920094c

pinsvcapi: extract types into their own module for better re-use later

view details

Hector Sanjuan

commit sha c3233ccf11df80d5dd996ecd2f0b32ff74fe51db

Wire in pinsvcapi to ipfs-cluster-service

view details

push time in 5 days

push eventipfs/ipfs-cluster

Hector Sanjuan

commit sha af9313d4fec6b802547164d257beae5c841bf082

Remove test files that should have not been committed

view details

push time in 5 days

PR opened filecoin-project/lily

Update readme linking to the documentation website.

Things that have been deleted are covered there.

+9 -99

0 comment

1 changed file

pr created time in 6 days

create barnchfilecoin-project/lily

branch : feat/readme-link-docs

created branch time in 6 days