profile
viewpoint

ljedrz/lambda_calculus 25

A simple, zero-dependency implementation of the untyped lambda calculus in Safe Rust

ljedrz/blc 1

Binary lambda calculus

ljedrz/rust-misc 1

Generic stuff written in Rust

ljedrz/api 0

Promise and RxJS APIs around Polkadot and any Substrate-based chain RPC calls. It is dynamically generated based on what the Substrate runtime provides in terms of metadata. Full documentation & examples available

ljedrz/chalk 0

A PROLOG-ish interpreter written in Rust, intended eventually for use in the compiler

ljedrz/compiler-builtins 0

Porting `compiler-rt` intrinsics to Rust

ljedrz/datafrog 0

A lightweight Datalog engine in Rust

ljedrz/ena 0

An implementation of union-find / congruence-closure in Rust. Extracted from rustc for independent experimentation.

ljedrz/interledger-rs 0

An easy-to-use, high-performance Interledger implementation written in Rust

pull request commentrust-lang/rust

Introduce `LocalDefId` to `HirId` lookup table

@ecstatic-morse I wasn't sure how to proceed there; there was a follow-up PR, https://github.com/rust-lang/rust/pull/65837, but it was closed due to inactivity.

marmeladema

comment created time in 2 days

Pull request review commentrust-lang/rust

Remove all uses of `NodeId` in `ResolverOutputs`

 impl Definitions {      #[inline]     pub fn local_def_id(&self, node: ast::NodeId) -> LocalDefId {-        self.opt_local_def_id(node).unwrap()+        self.opt_local_def_id(node).unwrap_or_else(|| {+            panic!("no entry for node id: `{:?}` / `{:?}`", node, self.opt_node_id_to_hir_id(node))

Ah, you're right, I forgot about that :+1:.

marmeladema

comment created time in 6 days

Pull request review commentrust-lang/rust

Remove all uses of `NodeId` in `ResolverOutputs`

 impl<'a> Resolver<'a> {         ResolverOutputs {             definitions: self.definitions.clone(),             cstore: Box::new(self.cstore().clone()),-            extern_crate_map: self.extern_crate_map.clone(),-            export_map: self.export_map.clone(),-            trait_map: self.trait_map.clone(),-            glob_map: self.glob_map.clone(),-            maybe_unused_trait_imports: self.maybe_unused_trait_imports.clone(),-            maybe_unused_extern_crates: self.maybe_unused_extern_crates.clone(),+            extern_crate_map: self+                .extern_crate_map+                .iter()+                .map(|(k, v)| (self.definitions.local_def_id(k.clone()).to_def_id(), v.clone()))

nit: cloned() after iter() should allow us to drop the individual clones in the maps; also applies to the few other instances below

marmeladema

comment created time in 6 days

Pull request review commentrust-lang/rust

Remove all uses of `NodeId` in `ResolverOutputs`

 impl Definitions {      #[inline]     pub fn local_def_id(&self, node: ast::NodeId) -> LocalDefId {-        self.opt_local_def_id(node).unwrap()+        self.opt_local_def_id(node).unwrap_or_else(|| {+            panic!("no entry for node id: `{:?}` / `{:?}`", node, self.opt_node_id_to_hir_id(node))

unwrap_or_else(|| panic!(x)) is equivalent to expect(x)

marmeladema

comment created time in 6 days

issue commentrust-lang/rust

`HirId`-ification initiative

Yes, if we can change that one to use a different id then I think we could start looking at conditional creation of the NodeId > HirId map (so that it's only created for save_analysis and docs.

zackmdavis

comment created time in 7 days

create barnchljedrz/interledger-rs

branch : address_interactions

created branch time in 7 days

Pull request review commentrs-ipfs/rust-ipfs

Upgrade libp2p 0.19.0

 impl<TStrategy: Strategy> NetworkBehaviour for Bitswap<TStrategy> {     {         // TODO concat messages to same destination to reduce traffic.         if let Some(event) = self.events.pop_front() {-            if let NetworkBehaviourAction::SendEvent { peer_id, event } = event {+            if let NetworkBehaviourAction::NotifyHandler {+                peer_id,+                event,+                handler,+            } = event+            {                 match self.connected_peers.get_mut(&peer_id) {                     None => {-                        debug!("  requeueing send event to {}", peer_id.to_base58());+                        debug!("  requeueing send event to {}", peer_id);+                        // FIXME: I wonder if this should be

I'd make this FIXME a bit more descriptive

koivunej

comment created time in 8 days

Pull request review commentrs-ipfs/rust-ipfs

Upgrade libp2p 0.18.1

 impl NetworkBehaviour for SwarmApi {     fn addresses_of_peer(&mut self, peer_id: &PeerId) -> Vec<Multiaddr> {         log::trace!("addresses_of_peer {}", peer_id);         if let Some(addr) = self.connected_peers.get(peer_id).cloned() {

this one could be self.connected_peers.get(peer_id).cloned().unwrap_or_default() instead of an if let

koivunej

comment created time in 9 days

Pull request review commentrs-ipfs/rust-ipfs

Upgrade libp2p 0.18.1

 impl NetworkBehaviour for SwarmApi {     fn addresses_of_peer(&mut self, peer_id: &PeerId) -> Vec<Multiaddr> {         log::trace!("addresses_of_peer {}", peer_id);         if let Some(addr) = self.connected_peers.get(peer_id).cloned() {-            vec![addr]+            addr         } else {             Default::default()         }     } -    fn inject_connected(&mut self, peer_id: PeerId, cp: ConnectedPoint) {+    fn inject_connection_established(+        &mut self,+        peer_id: &PeerId,+        _id: &ConnectionId,+        cp: &ConnectedPoint,+    ) {+        // TODO: could be that the connection is not yet fully established at this point         log::trace!("inject_connected {} {:?}", peer_id.to_string(), cp);-        let addr = match cp {-            ConnectedPoint::Dialer { address } => address,-            ConnectedPoint::Listener { send_back_addr, .. } => send_back_addr,-        };-        let conn = Connection {-            peer_id: peer_id.clone(),-            address: addr.clone(),-            rtt: None,-        };+        let addr = connection_point_addr(cp);         self.peers.insert(peer_id.clone());-        self.connected_peers.insert(peer_id, addr.clone());-        self.connections.insert(addr.clone(), conn);+        let connections = self+            .connected_peers+            .entry(peer_id.clone())+            .or_insert_with(Vec::new);++        connections.push(addr.clone());++        self.connections.insert(addr.clone(), peer_id.clone());         self.connect_registry.finish_subscription(&addr, Ok(()));     } -    fn inject_disconnected(&mut self, peer_id: &PeerId, cp: ConnectedPoint) {+    fn inject_connected(&mut self, _peer_id: &PeerId) {+        // we have at least one fully open connection and handler is running+    }++    fn inject_connection_closed(+        &mut self,+        peer_id: &PeerId,+        _id: &ConnectionId,+        cp: &ConnectedPoint,+    ) {         log::trace!("inject_disconnected {} {:?}", peer_id.to_string(), cp);-        let addr = match cp {-            ConnectedPoint::Dialer { address } => address,-            ConnectedPoint::Listener { send_back_addr, .. } => send_back_addr,+        let closed_addr = connection_point_addr(cp);+        let became_empty = if let Some(connections) = self.connected_peers.get_mut(peer_id) {+            if let Some(index) = connections.iter().position(|addr| addr == closed_addr) {+                connections.swap_remove(index);+            }+            connections.is_empty()+        } else {+            false         };-        self.connected_peers.remove(peer_id);-        self.connections.remove(&addr);+        if became_empty {+            self.connected_peers.remove(peer_id);+        }+        self.connections.remove(closed_addr);+        // FIXME: should be an error+        self.connect_registry+            .finish_subscription(closed_addr, Ok(()));     } -    fn inject_node_event(&mut self, _peer_id: PeerId, _event: void::Void) {}+    fn inject_disconnected(&mut self, peer_id: &PeerId) {+        for address in self+            .connected_peers+            .remove(peer_id)+            .into_iter()+            .flat_map(|i| i.into_iter())

.flatten()

koivunej

comment created time in 9 days

Pull request review commentrs-ipfs/rust-ipfs

Upgrade libp2p 0.18.1

 impl NetworkBehaviour for SwarmApi {     fn addresses_of_peer(&mut self, peer_id: &PeerId) -> Vec<Multiaddr> {         log::trace!("addresses_of_peer {}", peer_id);         if let Some(addr) = self.connected_peers.get(peer_id).cloned() {-            vec![addr]+            addr         } else {             Default::default()         }     } -    fn inject_connected(&mut self, peer_id: PeerId, cp: ConnectedPoint) {+    fn inject_connection_established(+        &mut self,+        peer_id: &PeerId,+        _id: &ConnectionId,+        cp: &ConnectedPoint,+    ) {+        // TODO: could be that the connection is not yet fully established at this point         log::trace!("inject_connected {} {:?}", peer_id.to_string(), cp);-        let addr = match cp {-            ConnectedPoint::Dialer { address } => address,-            ConnectedPoint::Listener { send_back_addr, .. } => send_back_addr,-        };-        let conn = Connection {-            peer_id: peer_id.clone(),-            address: addr.clone(),-            rtt: None,-        };+        let addr = connection_point_addr(cp);         self.peers.insert(peer_id.clone());-        self.connected_peers.insert(peer_id, addr.clone());-        self.connections.insert(addr.clone(), conn);+        let connections = self+            .connected_peers+            .entry(peer_id.clone())+            .or_insert_with(Vec::new);

or_default()

koivunej

comment created time in 9 days

Pull request review commentrs-ipfs/rust-ipfs

Upgrade libp2p 0.18.1

 pub struct SwarmApi {     events: VecDeque<NetworkBehaviourAction>,     peers: HashSet<PeerId>,     connect_registry: SubscriptionRegistry<Multiaddr, Result<(), String>>,-    connections: HashMap<Multiaddr, Connection>,-    connected_peers: HashMap<PeerId, Multiaddr>,+    connections: HashMap<Multiaddr, PeerId>,+    stats: HashMap<PeerId, Duration>,

nit: this name could be a bit more informative

koivunej

comment created time in 9 days

Pull request review commentrs-ipfs/rust-ipfs

Upgrade libp2p 0.18.1

 impl<TStrategy: Strategy> NetworkBehaviour for Bitswap<TStrategy> {         Vec::new()     } -    fn inject_connected(&mut self, peer_id: PeerId, cp: ConnectedPoint) {-        debug!("bitswap: inject_connected");-        debug!("  peer_id: {}", peer_id.to_base58());-        debug!("  connected_point: {:?}", cp);+    fn inject_connected(&mut self, peer_id: &PeerId) {+        debug!("bitswap: inject_connected {}", peer_id);         let ledger = Ledger::new();         self.connected_peers.insert(peer_id.clone(), ledger);-        self.send_want_list(peer_id);-        debug!("");+        self.send_want_list(peer_id.clone());

this clone() is not needed

koivunej

comment created time in 9 days

push eventljedrz/interledger-rs

ljedrz

commit sha 2d7ded580af8e73a5935b6dd3ef7fe6c1295f187

perf: remove a few redundant clones Signed-off-by: ljedrz <ljedrz@gmail.com>

view details

ljedrz

commit sha 51afc32caf62feb9a09fd53c95756af5143ed9b6

perf: make some u8-sized enums copiable Signed-off-by: ljedrz <ljedrz@gmail.com>

view details

push time in 12 days

PR opened interledger-rs/interledger-rs

Reviewers
A few drive-bys more

A few additional tweaks targeting unnecessary clones.

+11 -12

0 comment

5 changed files

pr created time in 12 days

create barnchljedrz/interledger-rs

branch : drive_bys

created branch time in 12 days

delete branch ljedrz/interledger-rs

delete branch : bits_and_bobs

delete time in 12 days

PR opened interledger-rs/interledger-rs

Reviewers
perf: remove a few clones of the incoming service chain

Address a bit of #615 by removing 2 unnecessary allocations of the incoming service.

+2 -4

0 comment

2 changed files

pr created time in 12 days

create barnchljedrz/interledger-rs

branch : partially_fix_615

created branch time in 12 days

push eventljedrz/interledger-rs

ljedrz

commit sha c97e727b77a70a2f5868520118a2b9c14e179769

make interledger-http build individually; remove unneeded calls to warp::Filter::boxed (#640) * fix: make interledger-http build individually Signed-off-by: ljedrz <ljedrz@gmail.com> * refactor: remove unneeded calls to warp::Filter::boxed Signed-off-by: ljedrz <ljedrz@gmail.com>

view details

push time in 12 days

PR opened interledger-rs/interledger-rs

Reviewers
Squash a few perf-related TODOs

These changes mostly target unnecessary allocations; individual commits describe specific changes.

In one case I changed the TODO to FIXME, because it's a bit of a liability; it can be easily fixed, but doing so requires 2 related tests to be adjusted:

---- balances_test::prepare_then_fulfill_with_settlement stdout ----
thread 'balances_test::prepare_then_fulfill_with_settlement' panicked at 'assertion failed: `(left == right)`
  left: `"No such file or directory (os error 2)"`,
 right: `"Broken pipe (os error 32)"`', crates/interledger-store/tests/redis/balances_test.rs:197:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- accounts_test::delete_accounts stdout ----
thread 'accounts_test::delete_accounts' panicked at 'assertion failed: `(left == right)`
  left: `"No such file or directory (os error 2)"`,
 right: `"Broken pipe (os error 32)"`', crates/interledger-store/tests/redis/accounts_test.rs:182:5

failures:
    accounts_test::delete_accounts
    balances_test::prepare_then_fulfill_with_settlement

which I am happy to do if it's desirable.

+21 -27

0 comment

8 changed files

pr created time in 12 days

create barnchljedrz/interledger-rs

branch : todo_hunting

created branch time in 12 days

pull request commentw3f/General-Grants-Program

Create offchain::ipfs speculative grant

Following Substrate's vision, it is not our intention to dictate how IPFS would be used. Same with voting - Substrate allows its users to tailor the consensus algorithm to their specific needs and we don't want to affect it. The goal is to enable DApp developers with the means to easily integrate IPFS into their custom pallets, which can offer a great host of functionalities, but are non-trivial to create. By integrating IPFS into the off-chain API we intend to make this process much simpler, which can benefit both the Substrate and IPFS ecosystems.

aphelionz

comment created time in 13 days

PR opened interledger-rs/interledger-rs

Reviewers
make interledger-http build individually; remove unneeded calls to warp::Filter::boxed
  • fix the individual build of the interledger-http crate that currently fails due to secrecy::SecretString requiring the alloc feature
  • remove calls to .boxed() where it is of no consequence to the return values of functions; they introduce extra indirection via Arc
+33 -58

0 comment

6 changed files

pr created time in 13 days

push eventljedrz/interledger-rs

ljedrz

commit sha 8d436278ac0ca94cacda2b1260268131bc0cbae6

fix: make interledger-http build individually Signed-off-by: ljedrz <ljedrz@gmail.com>

view details

ljedrz

commit sha 3309597a20fe9b1707e4d6bdaacedb4a193ebfe4

refactor: remove unneeded calls to warp::Filter::boxed Signed-off-by: ljedrz <ljedrz@gmail.com>

view details

push time in 13 days

create barnchljedrz/interledger-rs

branch : bits_and_bobs

created branch time in 13 days

fork ljedrz/interledger-rs

An easy-to-use, high-performance Interledger implementation written in Rust

http://interledger.rs

fork in 13 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+> Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), which "pins" content by default+* Unpin capabilities, which then would enable:+  * IPFS garbage collection of unpinned data+* Ability to bootstrap peers and join the global network+* Documentation, tests, and examples for the above++This could also lead parachain developers to do things like incentivizing IPFS content sharing (aka block providing), or joining a cluster.++## Milestone 3: Create reference implementation and community resources+>Estimated Timeframe: July 25 - August 25++Once the above work is done, `offchain::ipfs` will be functionally complete. In order to increase adoption and+foster community engagement, the grant team will provide thorough documentation, guidance and mentorship with+regards to implementing IPFS in a number of forms.++### Deliverables+* Finalized documentation, tests, and examples from Milestones #1 and #2+* A glossy web site documenting the API endpoints, perhaps in the “mdbook” format favored by the Rust community.+* A series of blog posts about `offchain::ipfs`+    * The problem statement and overall vision behind integrating IPFS to Web3 ecosystem +    * How `offchain::ipfs` solves the problem, how people can use it, and how implementing this gives you de-facto “parachain based IPFS clustering”+    * Best practices and future ideas+* A video demo and walkthrough of how this works+* Posting on aggregator sites like Hacker News and selected subreddits, and then being available to answer questions as they arise+* The aforementioned Docker image, including instructions on how to customize and rebuild the image.++This will benefit both the project and Substrate alike since all of the above items are potential for PR and+marketing, popularizing all of the technologies and organizations involved. One particular project that could benefit greatly from this effort is [Subsocial](http://subsocial.network/), who we are in active contact with.++# Risks++## #1. The Substrate ecosystem is evolving fast (and that’s a good thing)+Likelihood: High+Impact: Medium/High++The grant team plans on targeting version ^2.0.0 of Substrate. As this is currently a release candidate, everybody+in the ecosystem will need to be prepared for the “rug to be pulled out from beneath them” in terms of bugs,+regressions, and breaking changes. The grant team accepts this risk as part of a thriving open source ecosystem+in the web3 industry, and will mitigate this by following releases closely and communicating as frequently as+possible with our contacts both within web3 and the community at large.++## #2 `offchain::ipfs` will, of course, rely on rust-ipfs+Likelihood: Certain+Impact: Medium/Low++The highest level deliverable from this effort is the embedded IPFS node within a Substrate host. This refers specifically to https://github.com/ipfs-rust/rust-ipfs, which started as a community effort and is now being actively stewarded by Equilibrium.  As the effects of risk #1 start to materialize, updates to rust-ipfs will become necessary to either address Substrate version compatibility, or to take advantage of new Substrate features. Equilibrium acknowledges this risk and, as stated in the final section of this proposal, plans on maintaining this project as long as they are able.++## #3 Ensuring released dependencies happens on other projects’ timelines+Likelihood: Medium/Low+Impact: Very High++Rust-ipfs will need to rely on released dependencies in order to be published on crates.io and included in other Rust projects sustainably. There are a small (less than a handful) number of PRs and other such loose ends that will need to be resolved before the end of Milestone 1. However, the grant team cannot guarantee this outcome since we do not have direct control over the other projects. This risk can be mitigated - but not fully avoided - by proceeding along with Milestone 2 and 3 during any holding patterns that arise due to waiting for PR approvals.

perhaps "holding patterns" could be simplified to "hold-ups" or something similar, it doesn't sound very clear to me

aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+> Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), which "pins" content by default+* Unpin capabilities, which then would enable:+  * IPFS garbage collection of unpinned data+* Ability to bootstrap peers and join the global network+* Documentation, tests, and examples for the above++This could also lead parachain developers to do things like incentivizing IPFS content sharing (aka block providing), or joining a cluster.++## Milestone 3: Create reference implementation and community resources+>Estimated Timeframe: July 25 - August 25++Once the above work is done, `offchain::ipfs` will be functionally complete. In order to increase adoption and+foster community engagement, the grant team will provide thorough documentation, guidance and mentorship with+regards to implementing IPFS in a number of forms.++### Deliverables+* Finalized documentation, tests, and examples from Milestones #1 and #2+* A glossy web site documenting the API endpoints, perhaps in the “mdbook” format favored by the Rust community.+* A series of blog posts about `offchain::ipfs`+    * The problem statement and overall vision behind integrating IPFS to Web3 ecosystem +    * How `offchain::ipfs` solves the problem, how people can use it, and how implementing this gives you de-facto “parachain based IPFS clustering”+    * Best practices and future ideas+* A video demo and walkthrough of how this works+* Posting on aggregator sites like Hacker News and selected subreddits, and then being available to answer questions as they arise+* The aforementioned Docker image, including instructions on how to customize and rebuild the image.++This will benefit both the project and Substrate alike since all of the above items are potential for PR and+marketing, popularizing all of the technologies and organizations involved. One particular project that could benefit greatly from this effort is [Subsocial](http://subsocial.network/), who we are in active contact with.++# Risks++## #1. The Substrate ecosystem is evolving fast (and that’s a good thing)+Likelihood: High+Impact: Medium/High++The grant team plans on targeting version ^2.0.0 of Substrate. As this is currently a release candidate, everybody+in the ecosystem will need to be prepared for the “rug to be pulled out from beneath them” in terms of bugs,+regressions, and breaking changes. The grant team accepts this risk as part of a thriving open source ecosystem+in the web3 industry, and will mitigate this by following releases closely and communicating as frequently as+possible with our contacts both within web3 and the community at large.++## #2 `offchain::ipfs` will, of course, rely on rust-ipfs+Likelihood: Certain+Impact: Medium/Low++The highest level deliverable from this effort is the embedded IPFS node within a Substrate host. This refers specifically to https://github.com/ipfs-rust/rust-ipfs, which started as a community effort and is now being actively stewarded by Equilibrium.  As the effects of risk #1 start to materialize, updates to rust-ipfs will become necessary to either support address Substrate version compatibility, or to take advantage of new Substrate features. Equilibrium acknowledges this risk and, as stated in the final section of this proposal, plans on maintaining this project as long as they are able.++## #3 Ensuring released dependencies happens on other projects’ timelines

I'm not sure how to phrase this, but something is off wrt. grammar here

aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+> Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), which "pins" content by default+  * Unpin capabilities, which then would enable:+* IPFS garbage collection of unpinned data+* Ability to bootstrap peers and join the global network+* Documentation, tests, and examples for the above++This could also lead parachain developers to do things like incentivizing IPFS content sharing (aka block providing), or joining a cluster.++## Milestone 3: Create reference implementation and community resources+>Estimated Timeframe: July 25 - August 25++Once the above work is done, `offchain::ipfs` will be functionally complete. In order to increase adoption and+foster community engagement, the grant team will provide thorough documentation, guidance and mentorship with+regards to implementing IPFS in a number of forms.++### Deliverables+* Finalized documentation, tests, and examples from Milestones #1 and #2+* A glossy web site documenting the API endpoints, perhaps in the “mdbook” format favored by the Rust community.+* A series of blog posts about `offchain::ipfs`+    * The problem statement and overall vision behind integrating IPFS to Web3 ecosystem +    * How `offchain::ipfs` solves the problem and how can people use it, and how implementing this gives you de-facto “parachain based IPFS clustering”
    * How `offchain::ipfs` solves the problem, how people can use it, and how implementing this gives you de-facto “parachain based IPFS clustering”
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+> Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* [`ipfs.add`](https://docs.ipfs.io/reference/api/cli/#ipfs-add), which "pins" content by default+  * Unpin capabilities, which then would enable:
* Unpin capabilities, which then would enable:
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+> Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of `ipfs.add`, global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* ipfs.add, which "pins" content by default+  * Unpin capabilities, which then would enable:+* IPFS garbage collection of unpinned data
  * IPFS garbage collection of unpinned data
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+> Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of `ipfs.add`, global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* ipfs.add, which "pins" content by default+* Unpin capabilities, which then would enable:
  * Unpin capabilities, which then would enable:

apologies, I missed the fact it was a sub-point here

aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of ipfs.add, global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* ipfs add, which "pins" content by default+* Unpin capabilities, which then would enable:+  * IPFS garbage collection of unpinned data+* Ability to bootstrap peers and join the global network+* Documentation, tests, and examples for the above++This could also lead parachain developers to do things like incentivize IPFS content sharing (aka block providing), or joining a cluster.
This could also lead parachain developers to do things like incentivizing IPFS content sharing (aka block providing), or joining a cluster.
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of ipfs.add, global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* ipfs add, which "pins" content by default+* Unpin capabilities, which then would enable:+  * IPFS garbage collection of unpinned data
* IPFS garbage collection of unpinned data
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, `offchain::ipfs` would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate `offchain::ipfs` into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+so as not to conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++## Milestone 2: Unlock more Rust IPFS Features+Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of ipfs.add, global swarming, and+garbage collection would allow for parachain-based IPFS clustering. That is, anything that ends up on-chain could+potentially be pinned by any other nodes on that same parachain. This enables incredible benefits and only requires+the network to leverage its own nature and feature set.++Content would be requested via `offchain::ipfs`, or an off-chain HTTP call followed by `offchain::ipfs::add`. Then, the+off-chain worker would add the CIDs to the block as transactions.++### Deliverables:+* ipfs add, which "pins" content by default
* ipfs.add, which "pins" content by default
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, offchain::ipfs would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned (read: pre-1.0) Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate offchain::ipfs into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+as to not conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality+++## Milestone 2: Unlock more Rust IPFS Features+Estimated timeframe: June 25 - July 25++Once IPFS is firmly integrated into Substrate, we would like to take the time and refine the integration, both on the+Substrate and within rust-ipfs itself by adding features and hardening the existing code. ++While the work in Milestone 1 will be incredibly useful on its own, the addition of ipfs add, global swarming, and
While the work in Milestone 1 will be incredibly useful on its own, the addition of ipfs.add, global swarming, and
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, offchain::ipfs would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned (read: pre-1.0) Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate offchain::ipfs into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+as to not conflict+* Writing off-chain worker functions that make said requests to the on-host IPFS node+* Documentation, tests and examples of the delivered code and functionality++
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, offchain::ipfs would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned (read: pre-1.0) Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.++## Relevant Repositories+* https://github.com/ipfs-rust/rust-ipfs+* https://github.com/orbitdb/orbit-db+* https://github.com/ljedrz/lambda_calculus+* https://ambients.org++## Profiles / Resumes+* https://www.linkedin.com/in/joonas-koivunen-70273412/+* https://ipfs.io/ipfs/QmcHxD94cvJgq5ZZxQkEi7SRMwD5dBnkhQ3zzaVFqNWFJb+* https://www.linkedin.com/in/ljedrz/+* https://www.linkedin.com/in/vesa-ville-piiroinen-5b6b14/++# Development Roadmap++## Overall Schedule++The grant team will allot three months to this project with four team members allocated part time. If the work+started on May 25th and continued to August 25th it would total [65 working days](https://www.timeanddate.com/date/workdays.html?d1=25&m1=5&y1=2020&d2=25&m2=8&y2=2020&ti=on&).++Since this work will take place during the summer, the schedule might drift a bit due to vacations. Vacation plans+and schedule changes will be communicated clearly as soon as they are known.++## Docker Container++The grant team commits to delivering an OCI-compliant image of a reference implementation as part of the final+deliverable, for running the software in an OCI-runtime like Docker. This will help especially with the goal of+Milestone 3, since it will ease community adoption and provide more opportunities for education and mentoring as+well. Additionally, we will provide instructions on how a user may build their own image from the source code+with any modifications they need.++## Milestone 1: Integrate offchain::ipfs into Substrate+> Estimated timeframe: May 25 - June 25++In order to maximize impact and bring features to the Substrate community as quickly as possible, the grant team+will start by implementing [rust-ipfs](https://github.com/ipfs-rust/rust-ipfs/) “as-is” into Substrate core. As it requires a deeper integration than a typical+pallet might be able to provide, we would prefer this to be a PR against [paritytech/substrate](https://github.com/paritytech/substrate), but are open to+following Parity’s guidance as to the best way to integrate this.++This way Substrate users with off-chain workers enabled can immediately start testing and ultimately using the IPFS API.++### Deliverables:+* Integrating rust-ipfs on the Substrate host as a long-running process available to accept requests from the offchain-worker+* A libp2p integration, either by leveraging the the Substrate-internal libp2p, or by creating a separate integration+as to not conflict
so as not to conflict
aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, offchain::ipfs would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned (read: pre-1.0) Rust developer who has contributed not only core rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but also is a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.++Vesa-Ville is the Chief Research Officer at Equilibrium. As former CTO and co-founder of Haja Networks, he worked on the original theory, research and design of the [Ambients](https://github.com/ambientsprotocol/whitepaper), a protocol for distributing and executing deterministic programs safely in a decentralized, content-addressed network like IPFS. He has decades of experience in distributed systems, databases, and interoperability solutions. ++Together, we believe this team is uniquely qualified to complete this task, both in the implementation but also in communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.
Together, we believe this team is uniquely qualified to complete this task, both the implementation and communicating with the community in terms of the provided reference implementation and best practices when using, implementing, and interacting with IPFS in general.

minor grammar tweaks

aphelionz

comment created time in 19 days

Pull request review commenteqlabs/General-Grants-Program

Create offchain_ipfs.md

+# Project name+`offchain::ipfs`++# Project Description++Based on community feedback we received after announcing the Rust IPFS initiative, we would like to enable the+Substrate off-chain worker to make IPFS calls via a relay on the Substrate host. While this was at best cumbersome+and at worst infeasible for Substrate users before, the availability of a Rust-native IPFS implementation not only+makes this possible without external HTTP servers, but also optimizes network usage and performance significantly.++Once embedded, offchain::ipfs would enable common IPFS operations such as+* Storing data within IPFS via `ipfs.add`, `ipfs.dag.put`, and `ipfs.block.put`+* Retrieving data from IPFS via `ipfs.get`, `ipfs.dag.get` and `ipfs.block.get`+* Exchanging data with peers via `ipfs.bitswap`+* "Swarming" with the global IPFS network via libp2p++This would benefit the ecosystem by bringing together two mission-critical pieces of Web3 technology, allowing content+and data to be marshaled to and from Polkadot parachains, and improving availability in both networks. We believe that+this would be a valuable addition to Substrate and as such, we’d like to propose this to be available within the+official Substrate repository. Of course, we are willing to take Parity’s guidance on that.++While there are other implementations of IPFS (Go and JavaScript) we would prefer to natively embed IPFS in Rust to+keep the codebase consistent and leverage the performance and resource utilization benefits that Rust provides.++# About the Team++## Company+[Equilibrium](https://equilibrium.co)++## Members+* Mark Henderson - Rust Developer, Project Manager+* Vesa-Ville Piiroinen - SME / Research Advisor+* Joonas Koivunen - Rust Developer+* Lukasz Jedrzejczyk - Rust Developer++## Experience+Mark Henderson, in addition to being a [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=aphelionz), is also a [core contributor to OrbitDB](https://github.com/orbitdb/orbit-db/commits?author=aphelionz), which is a CRDT-based database system reliant on IPFS. For over a year, his full-time job was working on OrbitDB which, in addition to adding features and increasing performance in the distributed OrbitDB system, entailed both dealing personally with the idiosyncrasies of IPFS and also helping community members do the same.++Lukasz Jedrzejczyk has followed the development of Rust since before it became stable in 2015 and has been a proactive contributor to the [Rust compiler](https://github.com/rust-lang/rust/pulls?q=is%3Apr+sort%3Aupdated-desc+author%3Aljedrz+is%3Amerged) and its [StackOverflow community](https://stackoverflow.com/search?q=user:1870153+[rust]). He has participated in the creation of the Concordium blockchain and its ecosystem, concentrating on its client's network capabilities.++Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned (read: pre-1.0) Rust developer who has contributed not only core rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but also is a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.
Joonas Koivunen is an [Rust IPFS core contributor](https://github.com/ipfs-rust/rust-ipfs/commits?author=koivunej) and was instrumental in delivering the functionality proposed in a recent Protocol Labs devgrant. He is a seasoned (read: pre-1.0) Rust developer who has contributed to not only core Rust tooling like [cargo](https://github.com/rust-lang/cargo/pull/3924) and [clippy](https://github.com/rust-lang/rust-clippy/pull/1967), but is also a contributor to projects that Rust IPFS depends on, such as `rust-libp2p` and `aes-ctr`.

minor improvements to grammar

aphelionz

comment created time in 19 days

startedValveSoftware/Proton

started time in 25 days

push eventljedrz/substrate-node-template

ljedrz

commit sha 521002377497122a5e33b599fcdc6d52ccb63e1e

store the proof

view details

ljedrz

commit sha 2734dd71b93bfbc0352176ef9ac9e05fa3ffcb25

check if the magic tx is the first one

view details

push time in a month

create barnchljedrz/substrate-front-end-template

branch : proof_hash_friendly

created branch time in a month

push eventljedrz/substrate-node-template

ljedrz

commit sha 03753bc872da2c486938f894fac7511b534991e1

burn the extra issuance if the process fails

view details

push time in a month

push eventljedrz/substrate-node-template

ljedrz

commit sha ee809774d1b3085a22a6b81649116ad2b2459e09

craft a custom zero-cost weight for give_me_money

view details

ljedrz

commit sha 7e712fcf9e054477c742f47b76cc05a481455c4e

create the source account if non-existent

view details

push time in a month

push eventljedrz/substrate-node-template

ljedrz

commit sha a2b65d374ccf8db74076cef42950271b2da416f0

tweak Event

view details

push time in a month

push eventljedrz/substrate-node-template

ljedrz

commit sha 064a92e94f1774b655964f6846c5325052ecf146

make give_me_money free

view details

push time in a month

create barnchljedrz/substrate-node-template

branch : cross_palet_trickery

created branch time in a month

create barnchljedrz/substrate-node-template

branch : extra_pallet_tests

created branch time in a month

fork ljedrz/substrate-node-template

A new FRAME-based Substrate node, ready for hacking.

fork in a month

create barnchljedrz/api

branch : batch_tx_test

created branch time in a month

fork ljedrz/substrate

Substrate: The platform for blockchain innovators

fork in a month

fork ljedrz/api

Promise and RxJS APIs around Polkadot and any Substrate-based chain RPC calls. It is dynamically generated based on what the Substrate runtime provides in terms of metadata. Full documentation & examples available

https://polkadot.js.org/api/

fork in a month

starteddomgetter/NCoC

started time in a month

delete branch ljedrz/rust-1

delete branch : mir_inline_span_for_optimized_mir

delete time in a month

push eventljedrz/rust-1

ljedrz

commit sha 3c455fe8e1712af5996232053cd6ea3ea743a487

MIR: use span instead of NodeId to determine if optimized_mir should be run

view details

push time in a month

push eventljedrz/rust-1

ljedrz

commit sha 144ca6bcd44e94231a4bd84d4dd68fc77aa3f003

MIR: use span instead of NodeId to determine if optimized_mir should be run

view details

push time in a month

pull request commentrust-lang/rust

MIR: use Span instead of NodeId for optimized_mir purposes

If the order can be arbitrary then it makes sense to just use HirIds - I'll run the tests. Wouldn't this mean it can work in incremental builds too? HirIds are stable after all.

ljedrz

comment created time in a month

Pull request review commentrust-lang/rust

MIR: use Span instead of NodeId for optimized_mir purposes

 impl Inliner<'tcx> {                     continue;                 } -                let self_node_id = self.tcx.hir().as_local_node_id(self.source.def_id()).unwrap();-                let callee_node_id = self.tcx.hir().as_local_node_id(callsite.callee);+                let callee_hir_id = self.tcx.hir().as_local_hir_id(callsite.callee); -                let callee_body = if let Some(callee_node_id) = callee_node_id {+                let callee_body = if let Some(callee_hir_id) = callee_hir_id {+                    let self_hir_id = self.tcx.hir().as_local_hir_id(self.source.def_id()).unwrap();                     // Avoid a cycle here by only using `optimized_mir` only if we have-                    // a lower node id than the callee. This ensures that the callee will-                    // not inline us. This trick only works without incremental compilation.-                    // So don't do it if that is enabled.-                    if !self.tcx.dep_graph.is_fully_enabled()-                        && self_node_id.as_u32() < callee_node_id.as_u32()-                    {+                    // a lower span than the callee. This ensures that the callee will+                    // not inline us.+                    if self.tcx.hir().span(self_hir_id) < self.tcx.hir().span(callee_hir_id) {

Thanks, I reverted that bit.

ljedrz

comment created time in a month

push eventljedrz/rust-1

ljedrz

commit sha a270ba6e8ab1259b7759ec7cf5ea062bb8524a60

MIR: use span instead of NodeId to determine if optimized_mir should be run

view details

push time in a month

push eventljedrz/rust-1

ljedrz

commit sha 81d72839b08802ad0669c2a0f1aef18ad55bbf73

MIR: use span instead of NodeId to determine if optimized_mir should be run

view details

push time in a month

Pull request review commentrust-lang/rust

MIR: use Span instead of NodeId for optimized_mir purposes

 impl Inliner<'tcx> {                     continue;                 } -                let self_node_id = self.tcx.hir().as_local_node_id(self.source.def_id()).unwrap();

It seems that this assignment can be moved into the if block below; I'll update in a moment.

ljedrz

comment created time in a month

PR opened rust-lang/rust

MIR: use Span instead of NodeId for optimized_mir purposes

I wanted to see if I could limit the number of uses of NodeId when HirId is available and I saw that some of the MIR Inliner code could use Span instead of NodeId, not unlike in https://github.com/rust-lang/rust/pull/71197.

If I'm understanding the reason for not calling optimized_mir in incremental builds here correctly, this change could also allow us to do so.

This change could have perf implications.

+6 -9

0 comment

1 changed file

pr created time in a month

create barnchljedrz/rust-1

branch : mir_inline_span_for_optimized_mir

created branch time in a month

issue commentrust-lang/rust

Pretty serious perf regressions in opt benchmarks

Yes, seems to be the same thing; feel free to close this issue once it's confirmed, I seem to have missed the previous one :stuck_out_tongue: .

ljedrz

comment created time in a month

issue openedrust-lang/rust

Pretty serious perf regressions in opt benchmarks

It appears that one of the changes today caused serious regressions in some of the opt benches, mostly patched incremental:

regex-opt        | avg: 61.3% | min: -0.0% | max: 330.1%
webrender-opt    | avg: 78.2% | min: -0.0% | max: 312.6%
piston-image-opt | avg: 73.3% | min: -0.4% | max: 293.4%
ripgrep-opt      | avg: 31.3% | min: -0.1% | max: 125.3%

created time in a month

delete branch ljedrz/rust-1

delete branch : unsafe_unused

delete time in a month

Pull request review commentrust-lang/rust

Don't use the HirId to NodeId map in MIR

 pub fn check_unsafety(tcx: TyCtxt<'_>, def_id: DefId) {         }     } -    let mut unsafe_blocks: Vec<_> = unsafe_blocks.iter().collect();-    unsafe_blocks.sort_by_cached_key(|(hir_id, _)| tcx.hir().hir_id_to_node_id(*hir_id));-    let used_unsafe: FxHashSet<_> =-        unsafe_blocks.iter().flat_map(|&&(id, used)| used.then_some(id)).collect();-    for &(block_id, is_used) in unsafe_blocks {-        if !is_used {-            report_unused_unsafe(tcx, &used_unsafe, block_id);+    let (mut unsafe_used, mut unsafe_unused): (FxHashSet<_>, Vec<_>) = Default::default();+    for &(block_id, is_used) in unsafe_blocks.iter() {+        if is_used {+            unsafe_used.insert(block_id);+        } else {+            unsafe_unused.push(block_id);         }     }+    unsafe_unused.sort_by_cached_key(|hir_id| tcx.hir().span(*hir_id));

Done :+1: .

ljedrz

comment created time in a month

push eventljedrz/rust-1

ljedrz

commit sha 66575c9962188aa2f16df04fa1fef1786f0d216f

comment on the sorting of unused unsafe blocks

view details

push time in a month

Pull request review commentrust-lang/rust

Don't use the HirId to NodeId map in MIR

 pub fn check_unsafety(tcx: TyCtxt<'_>, def_id: DefId) {         }     } -    let mut unsafe_blocks: Vec<_> = unsafe_blocks.iter().collect();-    unsafe_blocks.sort_by_cached_key(|(hir_id, _)| tcx.hir().hir_id_to_node_id(*hir_id));-    let used_unsafe: FxHashSet<_> =-        unsafe_blocks.iter().flat_map(|&&(id, used)| used.then_some(id)).collect();-    for &(block_id, is_used) in unsafe_blocks {-        if !is_used {-            report_unused_unsafe(tcx, &used_unsafe, block_id);+    let (mut unsafe_used, mut unsafe_unused): (FxHashSet<_>, Vec<_>) = Default::default();+    for &(block_id, is_used) in unsafe_blocks.iter() {+        if is_used {+            unsafe_used.insert(block_id);+        } else {+            unsafe_unused.push(block_id);         }     }+    unsafe_unused.sort_by_cached_key(|hir_id| tcx.hir().span(*hir_id));

Sure, let me just run the tests to include their names.

ljedrz

comment created time in a month

Pull request review commentrust-lang/rust

Don't use the HirId to NodeId map in MIR

 pub fn check_unsafety(tcx: TyCtxt<'_>, def_id: DefId) {         }     } -    let mut unsafe_blocks: Vec<_> = unsafe_blocks.iter().collect();-    unsafe_blocks.sort_by_cached_key(|(hir_id, _)| tcx.hir().hir_id_to_node_id(*hir_id));-    let used_unsafe: FxHashSet<_> =-        unsafe_blocks.iter().flat_map(|&&(id, used)| used.then_some(id)).collect();-    for &(block_id, is_used) in unsafe_blocks {-        if !is_used {-            report_unused_unsafe(tcx, &used_unsafe, block_id);+    let (mut unsafe_used, mut unsafe_unused): (FxHashSet<_>, Vec<_>) = Default::default();+    for &(block_id, is_used) in unsafe_blocks.iter() {+        if is_used {+            unsafe_used.insert(block_id);+        } else {+            unsafe_unused.push(block_id);         }     }+    unsafe_unused.sort_by_cached_key(|hir_id| tcx.hir().span(*hir_id));

Yes, it doesn't appear to be; without the sort two (if I recall correctly) tests with nested unsafe blocks fail due to unaligned error messages.

ljedrz

comment created time in a month

issue commentrust-lang/rust

`HirId`-ification initiative

It appears that some of the remaining progress is blocked on https://github.com/rust-lang/rust/issues/71104.

zackmdavis

comment created time in a month

PR opened rust-lang/rust

Don't use the HirId to NodeId map in MIR

Another step towards not having to build a HirId to NodeId map other than for doc and RLS purposes.

We are currently sorting unsafe blocks by NodeId in check_unsafety; change it to sorting by Span instead; this passes the tests, but better ideas are welcome.

In addition, simplify the split between the used and unused unsafe blocks for readability and less sorting in the unused block vector.

cc https://github.com/rust-lang/rust/issues/50928

+11 -7

0 comment

1 changed file

pr created time in a month

create barnchljedrz/rust-1

branch : unsafe_unused

created branch time in a month

issue commentrust-lang/rust

`HirId`-ification initiative

For instance, we are still using the node_id_to_hir_id e.g. in middle and ty. If we were able to remove those, I think the only uses left would be in save_analysis and docs which, if I recall correctly, would mean we could not create the NodeId to HirId map when not building the docs or RLS.

zackmdavis

comment created time in 2 months

issue commentrust-lang/rust

`HirId`-ification initiative

It seems that not much has progressed since my last comment and there are still some uses of NodeIds that could be removed. When I have a little bit of time I'm going to attempt another pass; I'll open individual issues if I encounter trouble and require guidance so this can move forward.

zackmdavis

comment created time in 2 months

issue commentrust-lang/rust

`HirId`-ification initiative

@tshepang I haven't looked into it in a while, but I'll be happy to check the state of this initiative tomorrow.

zackmdavis

comment created time in 2 months

issue openedrust-lang/rust

A big performance regression in benchmarks

rustc performance data shows a pretty big regression between 74bd074e and 6050e52. I don't know if this is expected, but I'm putting this out there in case others see the dent in perf charts and wonder what had happened.

created time in 2 months

Pull request review commentrust-lang/rust

Make io::Read::read_to_end consider io::Take::limit

 impl<T: Read> Read for Take<T> {     unsafe fn initializer(&self) -> Initializer {         self.inner.initializer()     }++    fn read_to_end(&mut self, buf: &mut Vec<u8>) -> Result<usize> {+        let reservation_size = cmp::min(self.limit, 32) as usize;

This was not my choice - I only preserved the original value.

ljedrz

comment created time in 2 months

Pull request review commentrust-lang/rust

Make io::Read::read_to_end consider io::Take::limit

 fn append_to_string<F>(buf: &mut String, f: F) -> Result<usize> // avoid paying to allocate and zero a huge chunk of memory if the reader only // has 4 bytes while still making large reads if the reader does have a ton // of data to return. Simply tacking on an extra DEFAULT_BUF_SIZE space every-// time is 4,500 times (!) slower than this if the reader has a very small

As with the number 32, this was not my design choice; I was only fixing https://github.com/rust-lang/rust/issues/51746 - all the rest was left as-is.

ljedrz

comment created time in 2 months

Pull request review commentrust-lang/rust

Polymorphization

 impl DefPathBasedNames<'tcx> {                 let substs = substs.truncate_to(self.tcx, generics);                 self.push_generic_params(substs, iter::empty(), output, debug);             }+            ty::Param(_) => {+                output.push_str(&format!("{}", t));

Instead of format! you could use t.to_string(), which is faster.

davidtwco

comment created time in 2 months

push eventljedrz/rust-misc

ljedrz

commit sha 80523384be03ef6d025d06d3d07881bb4df32ba2

add the rolling hash differ

view details

push time in 2 months

push eventljedrz/rust-misc

ljedrz

commit sha 33611786e318103829d0b2bf87cecbc8c4124805

add the rolling hash differ

view details

push time in 2 months

push eventljedrz/rust-misc

ljedrz

commit sha e4c71d24cef69d474564bab7a3ca436713c0bae9

add the rolling hash differ

view details

push time in 2 months

more