profile
viewpoint

cryptonemo/bellman 1

zk-SNARK library

cryptonemo/asm-hashes 0

Assembly implementations of cryptographic hash functions

cryptonemo/bls-signatures 0

BLS Signatures in Rust

cryptonemo/ec-gpu 0

OpenCL code generator for finite-field arithmetic over arbitrary prime fields

cryptonemo/filecoin-ffi 0

C and CGO bindings for Filecoin's Rust libraries

cryptonemo/filecoin-phase2 0

Phase2 for Filecoin circuits

cryptonemo/filepath 0

Get the filesystem path of a file

push eventfilecoin-project/filecoin-ffi

nemo

commit sha 428503c87d917cc5e3e637983b43b4c260863bf0

fix: update deps to latest

view details

push time in an hour

push eventfilecoin-project/filecoin-ffi

nemo

commit sha fc4052c2db5e59baf094545e6f833831a42f0e7b

fix: update deps to latest

view details

push time in an hour

push eventfilecoin-project/rust-fil-proofs

fires

commit sha bae20c1d65f87eb0267ec0f5516330658d2cce33

reset consumer (#1535)

view details

nemo

commit sha 1923df52ec83c3458f293de08ef87214873d5f0d

feat: encode/decode/remove_data API with tests

view details

nemo

commit sha a1dc261878ac61cd0fa05d9332ee33e24395aac1

feat: add Jake's latest circuit code feat: added some filecoin-proofs types for updated porep_config

view details

nemo

commit sha 1cad3e6f205a0cbb81bc76c5c485294076852580

style: cargo fmt

view details

nemo

commit sha 0e43f37f36cadfe1057335f2dab1f352d9220c1d

feat: add a generate update proof stub API call

view details

nemo

commit sha d4afa914cf3b86d9e0b7d50d9b0084094c69f0d6

feat: start hooking up vanilla merkle proof generation for challenges

view details

nemo

commit sha 8a471046549b9a4354ecaf8dce8e7f72215eb22b

feat: add gathering of apex leaves fix: updates required after rebasing against master

view details

nemo

commit sha 2664f6f80d1107d4888e96ac1d7e1920a5feec9e

feat: formalize the vanilla update and vanilla proof types

view details

nemo

commit sha 8f7d4cff78bc51f8878ada7c2833fc2481e51944

fix: minor updates

view details

DrPeterVanNostrand

commit sha 076a447c8ac211e79d72d3b95c216dcc97221229

feat: empty-sector-update ProofScheme and CompoundProof

view details

nemo

commit sha 8cfa1ee1f64b3cc6b3b98a58408e4807b138fb81

feat: update filecoin-proofs with latest changes

view details

nemo

commit sha 66607a142091f5b8c805a3ee16eb2af0c462f5fb

fix: properly calculate phi using comm_r instead of comm_r_last feat: minor updates/cleanups

view details

DrPeterVanNostrand

commit sha 1355ef55b8d8c30f20877e5163f940c6e96a1e1d

refactor: remove lifetime parameter from EmptySectorUpdate struct

view details

nemo

commit sha 6b2e52aa615e59ceb00abe9158214cd04cbc8587

feat: complete vanilla proving and verify through tests

view details

nemo

commit sha ebcc3e77a82aaae6c0178015222575dc770982d9

feat: remove t_aux from being required in vanilla encode

view details

nemo

commit sha 3b5da8809a27e398c66fd96af71c9622b3c43b00

feat: remove t_aux from vanilla PrivateInputs

view details

DrPeterVanNostrand

commit sha 9eaab65b6e3981a85b5d6cca22401dd094170cd0

feat EmptySectorUpdateCompound tests

view details

DrPeterVanNostrand

commit sha 38f67e47ecc67a1794868a6bc6dd9ea081c8e3c8

style: clippy

view details

DrPeterVanNostrand

commit sha a66d3f05a3e6fec997a25c85a39945278b8b72a2

style: clippy

view details

DrPeterVanNostrand

commit sha d281d9ba15ffad3f81421e0b1acb591c7d4fcc15

fix: CompoundProof::circuit partition-index

view details

push time in 3 hours

push eventfilecoin-project/rust-fil-proofs

nemo

commit sha b557e748284c52b298c23bb84a9d508d1cc9b5b3

fix: ensure data is >= replica in remove data call

view details

push time in 3 hours

push eventfilecoin-project/rust-fil-proofs

fires

commit sha bae20c1d65f87eb0267ec0f5516330658d2cce33

reset consumer (#1535)

view details

push time in 7 hours

PR merged filecoin-project/rust-fil-proofs

Reviewers
reset consumer

Reset consumer every time when finish one layer.

If consumer not reset, may cause parent cache loading problem.

  1. cur_parent_ptr may point to a released memory location, causing panic.
  2. cur_parent_ptr may point to a wrong parent cache window, resulting wrong pc1 result.
+1 -0

5 comments

1 changed file

firesWu

pr closed time in 7 hours

issue commentfilecoin-project/rust-fil-proofs

Sporadic CI failures when running ignored tests in parallel

A recent example: https://app.circleci.com/pipelines/github/filecoin-project/rust-fil-proofs/3620/workflows/5fff99ac-0bad-4ba3-9b03-a08dcf511f56/jobs/82694

cryptonemo

comment created time in 7 hours

push eventfilecoin-project/rust-fil-proofs

nemo

commit sha b65a18b4c63cc6a9ed811f8fd37c55e55e7e075f

feat: persist p_aux and t_aux into cache dir after encoding/remove

view details

push time in a day

issue commentfilecoin-project/rust-fil-proofs

Add a rebuild_tree_c tool for restoring comm_c if needed

Where will you get these layers from if the sector is already sealed?

As mentioned, by extracting the layers (i.e. "unsealing").

Unless all of the layer data was saved, you would have to reseal. Both of those options seem prohibitive when compared with just retaining the 32-byte comm_c. I think that's the point: comm_c is truly 'persistent' auxiliary data and must be retained — because it cannot be cheaply recreated once replication is complete.

I understand your point, but on the contrary, I think of avoiding the expensive computation as the strongest reason it's cached in the first place.

cryptonemo

comment created time in a day

MemberEvent

issue commentfilecoin-project/filecoin-ffi

Building error on lotus v1.13.0 when FFI_BUILD_FROM_SOURCE

Try to use the build.sh script for building from source

mathshero

comment created time in 2 days

pull request commentfilecoin-project/rust-gpu-tools

feat: add core counts

Is the goal here eventually to move this out of bellman?

vmx

comment created time in 2 days

issue openedfilecoin-project/filecoin-ffi

Fix m1 Mac builds

Description

A build from source failure has been reported since the install script does not install the required cargo lipo tooling, whereas the CI scripting installs all required pre-reqs. This shouldn't affect systems that already have all of the pre-reqs, but shouldn't fail if they are not present.

created time in 2 days

issue openedfilecoin-project/rust-fil-proofs

Sporadic CI failures when running lifecycle tests in parallel

Description

We need to determine if this is an issue that has something to do with the test being run in parallel (i.e. corruption on parallel access to something), or if it's just a rare CI issue (such as sometimes running out of RAM/resources/etc).

As a point of comparison, this does not happen locally in testing.

created time in 2 days

issue commentfilecoin-project/rust-fil-proofs

You should not call "cache" something that stores persistent metadata

Renamed, as it's not a bug, but given that there is no standalone tool to regenerate the cached artifact, it's more like an enhancement/request to build that tool. See #1545 and feel free to give it a go if you'd like.

fai1coin

comment created time in 2 days

issue openedfilecoin-project/rust-fil-proofs

Add a rebuild_tree_c tool for restoring comm_c if needed

Description

Consider adding a standalone tool to rebuild tree_c from a sealed sector (by extracting the necessary layers and then re-building the tree).

This is related to #1542, as there is currently no standalone tool to restore the cached comm_c value if lost.

created time in 2 days

issue openedfilecoin-project/rust-fil-proofs

Add GPU support to the update_tree_r_last tool for faster tree building

Description

Currently the update_tree_r_last tool uses CPU tree rebuilding only. Recently, the tree_r building code was re-factored and made shareable outside of the crate, so this is now an option to consider.

created time in 2 days

push eventfilecoin-project/filecoin-ffi

nemo

commit sha ce7083b3d187ec3bc41a68ab66567bd4f3be6dfc

feat: update to latest proofs commit

view details

push time in 3 days

push eventfilecoin-project/rust-fil-proofs

nemo

commit sha d3804ce4c5d07e55832b113b0be55f299e34d64c

fix: use proper node count

view details

push time in 3 days

push eventfilecoin-project/rust-fil-proofs

nemo

commit sha 8603acc319ddc7f5143c3e054e8946dff9557698

feat: add optional data mapping of a specified length

view details

push time in 3 days

push eventfilecoin-project/filecoin-ffi

nemo

commit sha b46438d0f9c5744bdee2c8dd5e6bae5559b162e2

feat: update proofs to latest commit

view details

push time in 3 days

push eventfilecoin-project/rust-fil-proofs

nemo

commit sha 02030bf7aa24491a8c517b4076608640c84b2546

style: fix clippy warning

view details

push time in 3 days

push eventfilecoin-project/rust-fil-proofs

nemo

commit sha d853f0cda2c0e1d89df4b298651fee118b22fe89

fix: use different method to build binary tree From the lotus side, additional data may be stored after the nodes we want in the tree, so using a different builder method will allow that to happen without a panic

view details

push time in 3 days

push eventfilecoin-project/rust-fil-proofs

nemo

commit sha 8da834ae75e283bcf1c491da95dfe4b79cfa978e

feat: consolidate CPU tree building method

view details

push time in 3 days

push eventfilecoin-project/filecoin-ffi

nemo

commit sha 776ea15545f5993465799c40d0f5415e5a079bf0

fix: use a more descriptive name of sector update vanilla proofs fns

view details

push time in 13 days

push eventfilecoin-project/filecoin-ffi

nemo

commit sha 9422542760e67c6f483874e9367d96101fb668a4

docs: update comments

view details

push time in 13 days

push eventfilecoin-project/filecoin-ffi

nemo

commit sha a9d6641b155aedd7ea1afb8b151167bd58d04abe

feat: add new API and types for sector update proving

view details

push time in 13 days

Pull request review commentfilecoin-project/rust-fil-proofs

Add support for Empty Sector Update proofs (for SnapDeals)

 impl<'a, Tree: 'static + MerkleTreeTrait, G: 'static + Hasher> StackedDrg<'a, Tr     }      #[cfg(any(feature = "cuda", feature = "opencl"))]-    fn generate_tree_r_last<TreeArity>(+    fn prepare_tree_r_data(+        source: &DiskStore<<Tree::Hasher as Hasher>::Domain>,+        data: Option<&mut Data<'_>>,+        start: usize,+        end: usize,+    ) -> Result<TreeRElementData<Tree>> {+        if Self::use_gpu_tree_builder() {+            use fr32::bytes_into_fr;++            let mut layer_bytes = vec![0u8; (end - start) * std::mem::size_of::<Fr>()];+            source+                .read_range_into(start, end, &mut layer_bytes)+                .expect("failed to read layer bytes");++            let encoded_data: Vec<_> = layer_bytes+                .into_par_iter()+                .chunks(std::mem::size_of::<Fr>())+                .map(|chunk| bytes_into_fr(&chunk).expect("Could not create Fr from bytes."))+                .zip(+                    data.expect("failed to unwrap data").as_mut()+                        [(start * NODE_SIZE)..(end * NODE_SIZE)]+                        .par_chunks_mut(NODE_SIZE),+                )+                .map(|(key, data_node_bytes)| {+                    let data_node =+                        <Tree::Hasher as Hasher>::Domain::try_from_bytes(data_node_bytes)+                            .expect("try_from_bytes failed");++                    let mut encoded_fr: Fr = key;+                    let data_node_fr: Fr = data_node.into();+                    encode_fr(&mut encoded_fr, data_node_fr);+                    let encoded_fr_repr = encoded_fr.to_repr();+                    data_node_bytes.copy_from_slice(AsRef::<[u8]>::as_ref(&encoded_fr_repr));++                    encoded_fr+                })+                .collect();++            Ok(TreeRElementData::FrList(encoded_data))+        } else {+            let encoded_data: Vec<<Tree::Hasher as Hasher>::Domain> = source+                .read_range(start..end)?+                .into_par_iter()+                .zip(+                    data.expect("failed to unwrap data").as_mut()+                        [(start * NODE_SIZE)..(end * NODE_SIZE)]+                        .par_chunks_mut(NODE_SIZE),+                )+                .map(|(key, data_node_bytes)| {+                    let data_node =+                        <Tree::Hasher as Hasher>::Domain::try_from_bytes(data_node_bytes)+                            .expect("try from bytes failed");++                    let key_elem =+                        <Tree::Hasher as Hasher>::Domain::try_from_bytes(&key.into_bytes())+                            .expect("failed to convert key");+                    let encoded_node =+                        encode::<<Tree::Hasher as Hasher>::Domain>(key_elem, data_node);+                    data_node_bytes.copy_from_slice(AsRef::<[u8]>::as_ref(&encoded_node));++                    encoded_node+                })+                .collect();++            Ok(TreeRElementData::ElementList(encoded_data))

The code is already located in the helper method, we just have 2 versions of it depending on GPU/no-GPU support

cryptonemo

comment created time in 13 days

PullRequestReviewEvent

Pull request review commentfilecoin-project/rust-fil-proofs

Add support for Empty Sector Update proofs (for SnapDeals)

 readme = "README.md" storage-proofs-core = { path = "../storage-proofs-core", version = "^10.0.0", default-features = false} storage-proofs-porep = { path = "../storage-proofs-porep", version = "^10.0.0", default-features = false } storage-proofs-post = { path = "../storage-proofs-post", version = "^10.0.0", default-features = false }+storage-proofs-update = { path = "../storage-proofs-update", version = "^10.0.0", default-features = false }

Thanks, thought I had them all previously, but didn't.

cryptonemo

comment created time in 13 days

more