profile
viewpoint
Lucas Saldanha lucassaldanha @ConsenSys New Zealand http://www.lucassaldanha.com I'm a passionate software developer. Always trying to learn something new and looking for innovative ways of doing things that I already know.

lucassaldanha/eth-net-intelligence-api 3

Ethereum Network Intelligence API

lucassaldanha/besu-private-txs 1

Besu Private Transaction Example

lucassaldanha/besu-three-nodes-example 1

Docker compose config for a network with 3 Besu nodes with privacy enabled

lucassaldanha/java8-lamda-and-streams-presentation 1

A sample project with source examples used in the Java 8 Lamda & Streams presentation

ConsenSys/animals 0

A repo used on a Git workshop

iikirilov/besu 0

An enterprise-grade Java-based, Apache 2.0 licensed Ethereum client

lucassaldanha/animals 0

A repo used on a Git workshop

lucassaldanha/besu 0

An enterprise-grade Java-based, Apache 2.0 licensed Ethereum client

lucassaldanha/besu-docs 0

Documentation for Hyperledger Besu enterprise-grade Java-based, Apache 2.0 licensed Ethereum client

lucassaldanha/besu-local-nodes 0

Config for a 3 node Besu network (with privacy and permissioning)

Pull request review commentPegaSysEng/web3js-eea

added extra utils to add/remove participant from group

+const fs = require("fs");+const path = require("path");+const Web3 = require("web3");+const EEAClient = require("../../src");++const { besu, orion } = require("../keys");+const { logMatchingGroup, createHttpProvider } = require("../helpers.js");++const node = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);+const params = JSON.parse(fs.readFileSync(path.join(__dirname, "params.json")));++async function run() {+  const { privacyGroupId } = params;++  const findResultWithAddedNode = await node.privx.findOnChainPrivacyGroup({+    addresses: [+      orion.node1.publicKey,+      orion.node2.publicKey,+      orion.node11.publicKey+    ]+  });+  console.log("Found privacy groups with added node:");+  logMatchingGroup(findResultWithAddedNode, privacyGroupId);++  const removeResult = await node.privx.removeFromPrivacyGroup({+    participant: orion.node11.publicKey,+    enclaveKey: orion.node1.publicKey,+    privateFrom: orion.node1.publicKey,+    privacyGroupId,+    privateKey: besu.node1.privateKey+  });+  console.log("Removed third participant from privacy group:");

NIT: change log msg to Removed participant <key> from privacy group <id>

macfarla

comment created time in 2 days

Pull request review commentPegaSysEng/web3js-eea

added extra utils to add/remove participant from group

+const fs = require("fs");+const path = require("path");+const Web3 = require("web3");+const EEAClient = require("../../src");++const { besu, orion } = require("../keys");+const { createHttpProvider } = require("../helpers.js");++const node = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);+const params = JSON.parse(fs.readFileSync(path.join(__dirname, "params.json")));++async function run() {+  const { privacyGroupId } = params;++  const addResult = await node.privx.addToPrivacyGroup({+    participants: [orion.node11.publicKey],+    enclaveKey: orion.node1.publicKey,+    privateFrom: orion.node1.publicKey,+    privacyGroupId,+    privateKey: besu.node1.privateKey+  });++  console.log("Added new node to privacy group:");

NIT: It would be good to log something with more context like: Added participant <key> to privacy group <id>

macfarla

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent

issue commentPegaSysEng/teku

java.lang.IllegalStateException: Blocking message pending 10000 for BLOCKING

From my superficial analysis this seems to be somehow related to the way that RemoteEndpoint#sendString() is synchronous, and things endup queueing up if there are many concurrent sends. This is being used in the RemoteValidatorApi class.

One possible solution could be changing it to use RemoveEndpoint##sendStringByFuture(), which is more like a "send-and-forget" approach.

I believe for our use case it might make sense to use this send and forget approach. But I'm not 100% sure.

tzapu

comment created time in 2 days

issue commentPegaSysEng/teku

java.lang.IllegalStateException: Blocking message pending 10000 for BLOCKING

Seems related to this https://github.com/tipsy/javalin/issues/423. I heven't fully investigated it though.

tzapu

comment created time in 2 days

Pull request review commentPegaSysEng/web3js-eea

[MINOR] Ignore params

 coverage/  # temporary files example/eventLogs/params.json+example/multiTenancyExampleOnchainPrivacyEventLogs/params.json

We could replace lines 35 and 36 with something like **/params.json. But maybe, for now, we keep it simple :)

macfarla

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventlucassaldanha/besu-local-nodes

Sally MacFarlane

commit sha b92c8211beb4724f78b35a538e107755b2e09d7c

use allowlist Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

view details

Lucas Saldanha

commit sha ca047607d4da9e8fe8fcf117550e15f4ffa56a25

Merge pull request #4 from macfarla/allowlist use allowlist in configs

view details

push time in 2 days

Pull request review commentPegaSysEng/web3js-eea

added multi-tenancy examples

+const fs = require("fs");+const path = require("path");+const Web3 = require("web3");+const EEAClient = require("../../src");++const { besu, orion } = require("../keys");+const { createHttpProvider } = require("../helpers.js");++// use an orion key that is not a member of the group (eg orion.node11.jwt)+// to demonstrate that they can't create a subscription+const node = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);+const params = JSON.parse(fs.readFileSync(path.join(__dirname, "params.json")));++async function run() {+  const { privacyGroupId, contractAddress: address, blockNumber } = params;++  const filter = {+    address,+    fromBlock: blockNumber+  };++  console.log("Installing filter", filter);++  // Create subscription+  return node.priv+    .subscribe(privacyGroupId, filter, (error, result) => {+      if (!error) {+        console.log("Installed filter", result);+      } else {+        console.error("Problem installing filter:", error);+        throw error;+      }+    })+    .then(async subscription => {+      // Add handlers for incoming events+      subscription+        .on("data", log => {+          if (log.result != null) {+            // Logs from subscription are nested in `result` key+            console.log("LOG =>", log.result);+          } else {+            console.log("LOG =>", log);+          }+        })+        .on("error", console.error);++      // Unsubscribe and disconnect on interrupt+      process.on("SIGINT", async () => {+        console.log("unsubscribing");+        await subscription.unsubscribe((error, success) => {+          if (!error) {+            console.log("Unsubscribed:", success);

NIT: Should this include the filter ID that we have subscribed?

macfarla

comment created time in 2 days

Pull request review commentPegaSysEng/web3js-eea

added multi-tenancy examples

+const Web3 = require("web3");+const fs = require("fs");+const path = require("path");+const EEAClient = require("../../src");++const { besu, orion } = require("../keys");+const { createHttpProvider } = require("../helpers.js");++const bytecode = fs.readFileSync(+  path.join(__dirname, "../solidity/EventEmitter/EventEmitter.bin")+);++const node = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);++async function run() {+  const enclaveKey = orion.node1.publicKey;+  const addresses = [orion.node1.publicKey, orion.node2.publicKey];++  // create privacy group+  const onChainPrivacyGroupCreationResult = await node.privx.createPrivacyGroup(+    {+      participants: addresses,+      enclaveKey: orion.node1.publicKey,+      privateFrom: orion.node1.publicKey,+      privateKey: besu.node1.privateKey+    }+  );+  console.log("CREATION RESULT");+  console.log(onChainPrivacyGroupCreationResult);++  await node.priv.getTransactionReceipt(+    onChainPrivacyGroupCreationResult.commitmentHash,+    orion.node1.publicKey+  );++  const { privacyGroupId } = onChainPrivacyGroupCreationResult;+  console.log(privacyGroupId);++  // console.log("Created privacy group", privacyGroupId);++  // deploy contract+  const deployReceipt = await node.eea+    .sendRawTransaction({+      data: `0x${bytecode}`,+      privateFrom: enclaveKey,+      privacyGroupId,+      privateKey: besu.node1.privateKey+    })+    .then(hash => {+      return node.priv.getTransactionReceipt(hash, enclaveKey);+    });++  const { contractAddress, blockNumber } = deployReceipt;+  console.log("deployed", contractAddress);

Same here, we could log: Deployed contract at address <address>

macfarla

comment created time in 2 days

Pull request review commentPegaSysEng/web3js-eea

added multi-tenancy examples

+const Web3 = require("web3");+const fs = require("fs");+const path = require("path");+const EEAClient = require("../../src");++const { besu, orion } = require("../keys");+const { createHttpProvider } = require("../helpers.js");++const bytecode = fs.readFileSync(+  path.join(__dirname, "../solidity/EventEmitter/EventEmitter.bin")+);++const node = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);++async function run() {+  const enclaveKey = orion.node1.publicKey;+  const addresses = [orion.node1.publicKey, orion.node2.publicKey];++  // create privacy group+  const onChainPrivacyGroupCreationResult = await node.privx.createPrivacyGroup(+    {+      participants: addresses,+      enclaveKey: orion.node1.publicKey,+      privateFrom: orion.node1.publicKey,+      privateKey: besu.node1.privateKey+    }+  );+  console.log("CREATION RESULT");+  console.log(onChainPrivacyGroupCreationResult);++  await node.priv.getTransactionReceipt(+    onChainPrivacyGroupCreationResult.commitmentHash,+    orion.node1.publicKey+  );++  const { privacyGroupId } = onChainPrivacyGroupCreationResult;+  console.log(privacyGroupId);++  // console.log("Created privacy group", privacyGroupId);

Do we remove the comments or do we remove the line? :)

macfarla

comment created time in 2 days

Pull request review commentPegaSysEng/web3js-eea

added multi-tenancy examples

+const fs = require("fs");+const path = require("path");+const Web3 = require("web3");+const EEAClient = require("../../src");++const { orion, besu } = require("../keys.js");+const { createHttpProvider } = require("../helpers.js");++const binary = fs.readFileSync(+  path.join(__dirname, "../solidity/EventEmitter/EventEmitter.bin")+);++const node1 = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);++const createEventEmitterContract = privacyGroupId => {+  const contractOptions = {+    data: `0x${binary}`,+    privateFrom: orion.node1.publicKey,+    privacyGroupId,+    privateKey: besu.node1.privateKey+  };+  return node1.eea.sendRawTransaction(contractOptions);+};++const getPrivateContractAddress = transactionHash => {+  return node1.priv+    .getTransactionReceipt(transactionHash, orion.node1.publicKey)+    .then(privateTransactionReceipt => {+      return privateTransactionReceipt.contractAddress;+    });+};++module.exports = async () => {+  const privacyGroupCreationResult = await node1.privx.createPrivacyGroup({+    participants: [orion.node1.publicKey, orion.node2.publicKey],+    enclaveKey: orion.node1.publicKey,+    privateFrom: orion.node1.publicKey,+    privateKey: besu.node1.privateKey+  });++  console.log(privacyGroupCreationResult);

Should probably log something like: Created privacy group <ID>

macfarla

comment created time in 2 days

Pull request review commentPegaSysEng/web3js-eea

added multi-tenancy examples

+const Web3 = require("web3");+const EEAClient = require("../../src");++const { orion, besu } = require("../keys.js");+const { logMatchingGroup, createHttpProvider } = require("../helpers.js");++const node1 = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);+const node2 = new EEAClient(+  new Web3(createHttpProvider(orion.node2.jwt, besu.node2.url)),+  2018+);++module.exports = async () => {+  const onChainPrivacyGroupCreationResult = await node1.privx.createPrivacyGroup(+    {+      participants: [orion.node1.publicKey, orion.node2.publicKey],+      enclaveKey: orion.node1.publicKey,+      privateFrom: orion.node1.publicKey,+      privateKey: besu.node1.privateKey+    }+  );+  console.log("CREATION RESULT");

NIT: I think this is the only log all upper-case.

macfarla

comment created time in 2 days

Pull request review commentPegaSysEng/web3js-eea

added multi-tenancy examples

+const Web3 = require("web3");+const fs = require("fs");+const path = require("path");+const EEAClient = require("../../src");++const { besu, orion } = require("../keys");+const { createHttpProvider } = require("../helpers.js");++const bytecode = fs.readFileSync(+  path.join(__dirname, "../solidity/EventEmitter/EventEmitter.bin")+);++const node = new EEAClient(+  new Web3(createHttpProvider(orion.node1.jwt, besu.node1.url)),+  2018+);++async function run() {+  const enclaveKey = orion.node1.publicKey;+  const addresses = [orion.node1.publicKey, orion.node2.publicKey];++  // create privacy group+  const onChainPrivacyGroupCreationResult = await node.privx.createPrivacyGroup(+    {+      participants: addresses,+      enclaveKey: orion.node1.publicKey,+      privateFrom: orion.node1.publicKey,+      privateKey: besu.node1.privateKey+    }+  );+  console.log("CREATION RESULT");+  console.log(onChainPrivacyGroupCreationResult);++  await node.priv.getTransactionReceipt(+    onChainPrivacyGroupCreationResult.commitmentHash,+    orion.node1.publicKey+  );++  const { privacyGroupId } = onChainPrivacyGroupCreationResult;+  console.log(privacyGroupId);

I think it would be easier for the user if we log something like: "Created privacy group <id>"

Maybe this is what we were doing on line 42?

macfarla

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commenthyperledger/besu

handle case where onchain groups do not exist

 public void verifyPrivacyGroupContainsEnclavePublicKey(   public void verifyPrivacyGroupContainsEnclavePublicKey(       final String privacyGroupId, final String enclavePublicKey, final Optional<Long> blockNumber)       throws MultiTenancyValidationException {-    // TODO: There's potentially a bug here where an onchain privacyGroup-    // that isn't found will default to the offchain privacy group.-    final PrivacyGroup privacyGroup =-        onchainPrivacyGroupContract-            .flatMap(-                contract -> contract.getPrivacyGroupByIdAndBlockNumber(privacyGroupId, blockNumber))-            .orElse(enclave.retrievePrivacyGroup(privacyGroupId));--    if (!privacyGroup.getMembers().contains(enclavePublicKey)) {-      throw new MultiTenancyValidationException(-          "Privacy group must contain the enclave public key");+    Optional<PrivacyGroup> maybePrivacyGroup;+    PrivacyGroup offchainPrivacyGroup;+    if (onchainPrivacyGroupContract.isPresent()) {+      maybePrivacyGroup =+          onchainPrivacyGroupContract+              .get()+              .getPrivacyGroupByIdAndBlockNumber(privacyGroupId, blockNumber);+      if (maybePrivacyGroup.isEmpty()) {+        // group doesn't exist yet - this is normal for onchain privacy groups+      } else {+        // group DOES exist - does it contain the user?+        if (!maybePrivacyGroup.get().getMembers().contains(enclavePublicKey)) {+          throw new MultiTenancyValidationException(+              "Privacy group must contain the enclave public key");+        }

It feels like this logic could be polished a bit. Maybe separating the logic for finding a ground onchain vs offchain in separate methods would make it a bit easier to read?

macfarla

comment created time in 2 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventlucassaldanha/besu-local-nodes

Lucas Saldanha

commit sha f4b4b5ab55c1ec0ef5c39029a7641e20c66de120

Updated the versions of tested apps/libraries

view details

push time in 5 days

Pull request review commentPegaSysEng/permissioning-smart-contracts

Allow separate deployments

 const rulesContractName = Web3Utils.utf8ToHex("rules");  /* The address of the node ingress contract if pre deployed */ let nodeIngress = process.env.NODE_INGRESS_CONTRACT_ADDRESS;+let retainCurrentRulesContract = AllowlistUtils.getRetainNodeRulesContract();  module.exports = async(deployer, network) => {+    // exit early if we are NOT redeploying this contract+    if (retainCurrentRulesContract) {+        console.log("not deploying because retain=" + retainCurrentRulesContract);

same as above

macfarla

comment created time in 11 days

Pull request review commentPegaSysEng/permissioning-smart-contracts

Allow separate deployments

 This is the easiest way to get started for development with the permissioning Da ### Deploying the contracts 1. The [Besu documentation](https://besu.hyperledger.org/en/stable/Tutorials/Permissioning/Getting-Started-Onchain-Permissioning/)    describes how to use the contracts for onchain permissioning with Besu, including setting environment variables.--2. The following additional environment variables are optional and can be used to permit accounts and nodes during initial contract deployment+2. The following additional environment variables are optional and can be used to prevent redeployment of rules contracts. If set to true, that contract will not be redeployed and current list data will be preserved.+  - `RETAIN_ADMIN_CONTRACT=true`+  - `RETAIN_NODE_RULES_CONTRACT=true`+  - `RETAIN_ACCOUNT_RULES_CONTRACT=true`+3. The following additional environment variables are optional and can be used to permit accounts and nodes during initial contract deployment   - `INITIAL_ADMIN_ACCOUNTS`: The admin account addresses. Comma-separated multiple addresses can be specified   - `INITIAL_ALLOWLISTED_ACCOUNTS`: The permitted account addresses. Comma-separated multiple addresses can be specified   - `INITIAL_ALLOWLISTED_NODES`: The enode URLs of permitted nodes. Comma-separated multiple nodes can be specified-3. If this is the first time setting up the project, run `yarn install` to initialize project dependencies, otherwise skip this step-4. With these environment variables provided run `truffle migrate --reset` to deploy the contracts+4. If this is the first time setting up the project, run `yarn install` to initialize project dependencies, otherwise skip this step

Not related to this PR but a good FYI: In markdown, you can use 1. for every line in a numbered list and it will automatically calculate the right number (https://guides.github.com/features/mastering-markdown/#syntax).

This way you won't need to change the indexes every time you add/remove something :)

macfarla

comment created time in 11 days

Pull request review commentPegaSysEng/permissioning-smart-contracts

Allow separate deployments

 const rulesContractName = Web3Utils.utf8ToHex("rules");  /* The address of the account ingress contract if pre deployed */ let accountIngress = process.env.ACCOUNT_INGRESS_CONTRACT_ADDRESS;+let retainCurrentRulesContract = AllowlistUtils.getRetainAccountRulesContract();  module.exports = async(deployer, network) => {+    // exit early if we are NOT redeploying this contract+    if (retainCurrentRulesContract) {+        console.log("not deploying because retain=" + retainCurrentRulesContract);

same as above

macfarla

comment created time in 11 days

Pull request review commentPegaSysEng/permissioning-smart-contracts

Allow separate deployments

 const AllowlistUtils = require('../scripts/allowlist_utils'); const Admin = artifacts.require("./Admin.sol");  module.exports = async(deployer, network) => {+    // exit early if we are NOT redeploying this contract+    let retainCurrentRulesContract = AllowlistUtils.getRetainAdminContract();+    if (retainCurrentRulesContract) {+        console.log("not deploying because retain=" + retainCurrentRulesContract);

I think we could improve the msg specifying the contract that won't be deployed.

e.g.

[...] not deploying Admin Contract because [...]
macfarla

comment created time in 11 days

PullRequestReviewEvent

Pull request review commentPegaSysEng/permissioning-smart-contracts

Allow separate deployments

 This is the easiest way to get started for development with the permissioning Da ### Deploying the contracts 1. The [Besu documentation](https://besu.hyperledger.org/en/stable/Tutorials/Permissioning/Getting-Started-Onchain-Permissioning/)    describes how to use the contracts for onchain permissioning with Besu, including setting environment variables.--2. The following additional environment variables are optional and can be used to permit accounts and nodes during initial contract deployment+2. The following additional environment variables are optional and can be used to prevent redeployment of rules contracts. If set to true, that contract will not be redeployed and current list data will be preserved.

Should we mention the default behaviour?

macfarla

comment created time in 11 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

fork lucassaldanha/caliper

A blockchain benchmark framework to measure performance of multiple blockchain solutions https://wiki.hyperledger.org/display/caliper

https://hyperledger.github.io/caliper/

fork in 17 days

push eventPegaSysEng/web3js-eea

Sally MacFarlane

commit sha f1e6d5752deee0d1dea4c4f22c42dc10312b4470

[MINOR] added note about keys.js (#135) * added note about keys.js Signed-off-by: Sally MacFarlane <sally.macfarlane@consensys.net>

view details

push time in 17 days

PR merged PegaSysEng/web3js-eea

[MINOR] added note about keys.js

Signed-off-by: Sally MacFarlane sally.macfarlane@consensys.net

+2 -0

0 comment

2 changed files

macfarla

pr closed time in 17 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

push eventPegaSysEng/orion

Tim Beiko

commit sha 3feb09bc1b2ba7461c803f12346048fe4d32f0ff

1.6 Update (#389) Signed-off-by: Tim Beiko <t.beiko23@gmail.com>

view details

push time in 20 days

PR merged PegaSysEng/orion

1.6 Update

Signed-off-by: Tim Beiko t.beiko23@gmail.com

+15 -18

1 comment

2 changed files

timbeiko

pr closed time in 20 days

PullRequestReviewEvent

delete branch lucassaldanha/teku

delete branch : separate-validator

delete time in 24 days

push eventPegaSysEng/teku

Lucas Saldanha

commit sha 50cfa70b4ad694151014ddf063a32ac7e9aa7709

Running Validator Node independently (#2683) * Running Validator Node independently * Added validator client subcommand Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in 24 days

PR merged PegaSysEng/teku

Running Validator Node independently

PR Description

  • Options for starting only a Validator Node (connecting to a Beacon Node).

Here is an example of how to run a minimum beacon node and a validator client:

Beacon Node

teku --network minimal --data-path=/Users/lucas/Library/teku-beacon/data --Xinterop-enabled --rest-api-enabled --Xremote-validator-api-enabled

Validator Client

teku validator-client --network minimal --data-path=/Users/lucas/Library/teku-validator/data --log-include-validator-duties-enabled --Xinterop-enabled --Xinterop-owned-validator-count=64

New Properties

Beacon Node new properties:

  • --Xremote-validator-api-enabled
    • Enables the WebSocket server that the remote validator client can connect to in order to receive Beacon Node events (default: false)
  • --Xremote-validator-api-port <port>
    • The port that should be used to accept connection in the WebSocket server (default: 9999)
  • --Xremote-validator-api-interface <interface>
    • The network interface that the WebSocket service should bind to (default: 127.0.0.1)
  • --Xremote-validator-api-max-subscribers <integer>
    • The maximum number of remote validators that can connect to the Beacon Node (default: 1,000)

Validator Client

For the validator client, we have created a new subcommand validator-client. This subcommand has the following options:

  • --Xbeacon-node-api-endpoint <endpoint>
    • The endpoint of the Rest API service of the Beacon Node (default: http://127.0.0.1:5051)
  • --Xbeacon-node-events-endpoint <endpoint>
    • The endpoint of the WebSocket service of the Beacon Node (default: ws://127.0.0.1:9999)

Fixed Issue(s)

fixes #2450

Documentation

  • [X] I thought about documentation and added the documentation label to this PR if updates are required.
+469 -53

0 comment

16 changed files

lucassaldanha

pr closed time in 24 days

issue closedPegaSysEng/teku

Enable Validator Service to run independently

After preparing the validator service to consumer events from the Beacon Node via WebSockets and to interact with the Beacon Node via its REST API, we can have a standalone validator running.

closed time in 24 days

lucassaldanha

push eventlucassaldanha/teku

Adrian Sutton

commit sha b5c7d2012d464357a57b3730b78515b12c130dfe

Validate status update before accepting (#2712) Applies validation to all status updates, not just the first one.

view details

Anton Nashatyrev

commit sha 994997f8208ee0b74d7becaacda23c0d496d0a37

Enable gossip 1.1 flood publish option (#2717)

view details

Paul Harris

commit sha 6175077f3eac18f22e75b8725698891340820d8a

Added import and export for slashing protection data. (#2684) * Added import and export for slashing protection data. - teku slashing-protection export --data-path=<PATH> --to=<FILE> - teku slashing-protection import --data-path=<PATH> --from=<FILE> - --data-path defaults to the default path for teku. Exporting will only export to the minimal interchange format, as that is all that's stored by teku. On import, complete or minimal files are supported, and summarised for use by teku. If the genesis_validators_root does not match, the process will halt rather than corrupting existing data. If an existing file is found, the MAX of the existing data and imported data will be stored, to ensure there is no possibility of stepping 'backwards' in slashing protection data. fixes #2601 Signed-off-by: Paul Harris <paul.harris@consensys.net> Co-authored-by: Adrian Sutton <adrian@symphonious.net>

view details

Adrian Sutton

commit sha 3aabe4abee6b13cb8cea2485de4a46fada25a67a

Ensure constants are reset after tests that modify them. (#2723)

view details

mbaxter

commit sha aff52dc9b44c06c4a61af8e972bffd2aacc3d046

Separate initial state handling from interop handling (#2721)

view details

Adrian Sutton

commit sha 40d64cf41d67f3a2e21e2451d570aa6a18e44a95

Add security policy. (#2724)

view details

Paul Harris

commit sha 6c155094e2e67447fb0853313b05af0fb0ad26a2

Update existing /v1 endpoints to their standard path (#2725) - moved /v1/node/version to /eth/v1/node/version - moved /v1/node/identity to /eth/v1/node/identity - deprecated /node/version, existing consumers should update to use /eth/v1/node/version. Added integration tests for the standard endpoints to show they are able to be called and deserialised successfully. partially addresses #2722 Signed-off-by: Paul Harris <paul.harris@consensys.net>

view details

Lucas Saldanha

commit sha 5ac0dfc4ca00d1de1361c02e3089915c65d6848b

Merge branch 'master' into separate-validator Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

Lucas Saldanha

commit sha 6a2e25b84cfe832c62a419307dd22b970dd53797

Updated to use DataOptions only Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

Lucas Saldanha

commit sha f2d5dac3e11e402adfe07f1a2bb8a7bcdc6ca7e4

Created ServiceController abstract class Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in 24 days

Pull request review commentPegaSysEng/teku

Running Validator Node independently

+/*+ * Copyright 2019 ConsenSys AG.+ *+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with+ * the License. You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the+ * specific language governing permissions and limitations under the License.+ */++package tech.pegasys.teku.services;++import java.util.ArrayList;+import java.util.Iterator;+import java.util.List;+import tech.pegasys.teku.infrastructure.async.SafeFuture;+import tech.pegasys.teku.service.serviceutils.Service;+import tech.pegasys.teku.service.serviceutils.ServiceConfig;+import tech.pegasys.teku.validator.client.ValidatorClientService;++public class ValidatorNodeServiceController extends Service {++  private final List<Service> services = new ArrayList<>();++  public ValidatorNodeServiceController(final ServiceConfig config) {+    services.add(ValidatorClientService.create(config));+  }++  @Override+  protected SafeFuture<?> doStart() {+    final Iterator<Service> iterator = services.iterator();+    SafeFuture<?> startupFuture = iterator.next().start();+    while (iterator.hasNext()) {+      final Service nextService = iterator.next();+      startupFuture = startupFuture.thenCompose(__ -> nextService.start());+    }+    return startupFuture;+  }++  @Override+  protected SafeFuture<?> doStop() {+    return SafeFuture.allOf(services.stream().map(Service::stop).toArray(SafeFuture[]::new));+  }

Done

lucassaldanha

comment created time in 24 days

PullRequestReviewEvent

Pull request review commentPegaSysEng/teku

Running Validator Node independently

+/*+ * Copyright 2019 ConsenSys AG.+ *+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with+ * the License. You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the+ * specific language governing permissions and limitations under the License.+ */++package tech.pegasys.teku.cli.subcommand;++import com.google.common.base.Throwables;+import java.util.concurrent.Callable;+import java.util.concurrent.CompletionException;+import picocli.CommandLine.Command;+import picocli.CommandLine.Mixin;+import picocli.CommandLine.ParentCommand;+import tech.pegasys.teku.cli.BeaconNodeCommand;+import tech.pegasys.teku.cli.options.DataOptions;+import tech.pegasys.teku.cli.options.InteropOptions;+import tech.pegasys.teku.cli.options.LoggingOptions;+import tech.pegasys.teku.cli.options.MetricsOptions;+import tech.pegasys.teku.cli.options.NetworkOptions;+import tech.pegasys.teku.cli.options.ValidatorClientOptions;+import tech.pegasys.teku.cli.options.ValidatorOptions;+import tech.pegasys.teku.storage.server.DatabaseStorageException;+import tech.pegasys.teku.util.cli.PicoCliVersionProvider;+import tech.pegasys.teku.util.config.InvalidConfigurationException;+import tech.pegasys.teku.util.config.NetworkDefinition;+import tech.pegasys.teku.util.config.TekuConfiguration;++@Command(+    name = "validator-client",+    description = "Run a Validator Client that connects to a remote Beacon Node",+    showDefaultValues = true,+    abbreviateSynopsis = true,+    mixinStandardHelpOptions = true,+    versionProvider = PicoCliVersionProvider.class,+    synopsisHeading = "%n",+    descriptionHeading = "%nDescription:%n%n",+    optionListHeading = "%nOptions:%n",+    footerHeading = "%n",+    footer = "Teku is licensed under the Apache License 2.0")+public class ValidatorClientCommand implements Callable<Integer> {++  @Mixin(name = "Validator")+  private ValidatorOptions validatorOptions;++  @Mixin(name = "Validator Client")+  private ValidatorClientOptions validatorClientOptions;++  @Mixin(name = "Network")+  private NetworkOptions networkOptions;++  @Mixin(name = "Data")+  private DataOptions dataOptions;++  @Mixin(name = "Interop")+  private InteropOptions interopOptions;++  @Mixin(name = "Logging")+  private LoggingOptions loggingOptions;++  @Mixin(name = "Metrics")+  private MetricsOptions metricsOptions;++  @ParentCommand private BeaconNodeCommand parentCommand;++  @Override+  public Integer call() {+    try {+      parentCommand.setLogLevels();+      final TekuConfiguration tekuConfiguration = tekuConfiguration();+      parentCommand.getStartAction().accept(tekuConfiguration);+      return 0;+    } catch (InvalidConfigurationException | DatabaseStorageException ex) {+      parentCommand.reportUserError(ex);+    } catch (CompletionException e) {+      if (Throwables.getRootCause(e) instanceof InvalidConfigurationException) {+        parentCommand.reportUserError(Throwables.getRootCause(e));+      } else {+        parentCommand.reportUnexpectedError(e);+      }+    } catch (Throwable t) {+      parentCommand.reportUnexpectedError(t);+    }+    return 1;+  }++  private TekuConfiguration tekuConfiguration() {+    return TekuConfiguration.builder()+        .setNetwork(NetworkDefinition.fromCliArg(networkOptions.getNetwork()))+        .setInteropGenesisTime(interopOptions.getInteropGenesisTime())+        .setInteropOwnedValidatorStartIndex(interopOptions.getInteropOwnerValidatorStartIndex())+        .setInteropOwnedValidatorCount(interopOptions.getInteropOwnerValidatorCount())+        .setInteropNumberOfValidators(interopOptions.getInteropNumberOfValidators())+        .setInteropEnabled(interopOptions.isInteropEnabled())+        .setValidatorKeystoreFiles(validatorOptions.getValidatorKeystoreFiles())+        .setValidatorKeystorePasswordFiles(validatorOptions.getValidatorKeystorePasswordFiles())+        .setValidatorKeys(validatorOptions.getValidatorKeys())+        .setValidatorExternalSignerPublicKeys(+            validatorOptions.getValidatorExternalSignerPublicKeys())+        .setValidatorExternalSignerUrl(validatorOptions.getValidatorExternalSignerUrl())+        .setValidatorExternalSignerTimeout(validatorOptions.getValidatorExternalSignerTimeout())+        .setGraffiti(validatorOptions.getGraffiti())+        .setLogColorEnabled(loggingOptions.isLogColorEnabled())+        .setLogIncludeEventsEnabled(loggingOptions.isLogIncludeEventsEnabled())+        .setLogIncludeValidatorDutiesEnabled(loggingOptions.isLogIncludeValidatorDutiesEnabled())+        .setLogDestination(loggingOptions.getLogDestination())+        .setLogFile(loggingOptions.getLogFile())+        .setLogFileNamePattern(loggingOptions.getLogFileNamePattern())+        .setLogWireCipher(loggingOptions.isLogWireCipherEnabled())+        .setLogWirePlain(loggingOptions.isLogWirePlainEnabled())+        .setLogWireMuxFrames(loggingOptions.isLogWireMuxEnabled())+        .setLogWireGossip(loggingOptions.isLogWireGossipEnabled())+        .setMetricsEnabled(metricsOptions.isMetricsEnabled())+        .setMetricsPort(metricsOptions.getMetricsPort())+        .setMetricsInterface(metricsOptions.getMetricsInterface())+        .setMetricsCategories(metricsOptions.getMetricsCategories())+        .setMetricsHostAllowlist(metricsOptions.getMetricsHostAllowlist())+        .setDataPath(dataOptions.getDataPath())+        .setDataStorageMode(dataOptions.getDataStorageMode())+        .setDataStorageFrequency(dataOptions.getDataStorageFrequency())+        .setDataStorageCreateDbVersion(dataOptions.getCreateDbVersion())

Done :)

lucassaldanha

comment created time in 24 days

PullRequestReviewEvent

Pull request review commentPegaSysEng/teku

Running Validator Node independently

+/*+ * Copyright 2019 ConsenSys AG.+ *+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with+ * the License. You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the+ * specific language governing permissions and limitations under the License.+ */++package tech.pegasys.teku.cli.subcommand;++import com.google.common.base.Throwables;+import java.util.concurrent.Callable;+import java.util.concurrent.CompletionException;+import picocli.CommandLine.Command;+import picocli.CommandLine.Mixin;+import picocli.CommandLine.ParentCommand;+import tech.pegasys.teku.cli.BeaconNodeCommand;+import tech.pegasys.teku.cli.options.DataOptions;+import tech.pegasys.teku.cli.options.InteropOptions;+import tech.pegasys.teku.cli.options.LoggingOptions;+import tech.pegasys.teku.cli.options.MetricsOptions;+import tech.pegasys.teku.cli.options.NetworkOptions;+import tech.pegasys.teku.cli.options.ValidatorClientOptions;+import tech.pegasys.teku.cli.options.ValidatorOptions;+import tech.pegasys.teku.storage.server.DatabaseStorageException;+import tech.pegasys.teku.util.cli.PicoCliVersionProvider;+import tech.pegasys.teku.util.config.InvalidConfigurationException;+import tech.pegasys.teku.util.config.NetworkDefinition;+import tech.pegasys.teku.util.config.TekuConfiguration;++@Command(+    name = "validator-client",+    description = "Run a Validator Client that connects to a remote Beacon Node",+    showDefaultValues = true,+    abbreviateSynopsis = true,+    mixinStandardHelpOptions = true,+    versionProvider = PicoCliVersionProvider.class,+    synopsisHeading = "%n",+    descriptionHeading = "%nDescription:%n%n",+    optionListHeading = "%nOptions:%n",+    footerHeading = "%n",+    footer = "Teku is licensed under the Apache License 2.0")

Done!

lucassaldanha

comment created time in 24 days

PullRequestReviewEvent

Pull request review commentPegaSysEng/teku

Running Validator Node independently

 public TekuConfiguration build() {           "eth1-deposit-contract-address is required if eth1-endpoint is specified.");     } +    // TODO validate config for validator node only

Yeah, we don't need it :)

lucassaldanha

comment created time in 24 days

PullRequestReviewEvent

push eventlucassaldanha/teku

Cem Ozer

commit sha b14a8a8962e736690230a5c352b04ea6281f886e

Print warning when the Eth1 chain is shorter than follow distance (#2695)

view details

mbaxter

commit sha 16f6b576d0943f8a957c9ddb250245dece8ca756

[Issue-2566] Exit teku if eth1 service fails to start up (#2711)

view details

mbaxter

commit sha 53fa964e9c2642686862fcedcc65d8bc41ab07b0

[Issue 2566] Add deposit event validation (#2702)

view details

mbaxter

commit sha 4e5e61dd78e50849f64613976a4a781fbfc7b445

[Issue-2566] Additional validation for genesis deposits (#2705)

view details

Ben Edgington

commit sha 0cc8d4c3f2a70223b8b505926bea839a891819e1

Update default network to medalla (#2720)

view details

Paul Harris

commit sha b424c965a16b63b25ea7b04907b0c9a3bd0bd267

clean up recentChainData finalizedEpoch handling (#2714) - created getFinalizedCheckpoint returning an optional - removed getBestJustifiedEpoch as it wasn't used - removed getFinalizedRoot as it was used where the finalized checkpoint was already needed, so we now get the checkpoint and reference multiple attributes, rather than effectively fetching the finalizedCheckpoint multiple times. fixes #2398 Signed-off-by: Paul Harris <paul.harris@consensys.net>

view details

Lucas Saldanha

commit sha 4908ac4e79e172c610fa6614bc8e989904edc0ae

Merge branch 'master' into separate-validator Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in 24 days

push eventlucassaldanha/teku

Lucas Saldanha

commit sha a6ac6d07fe2e478a8c0150ebb09125d34d3d1e4a

Adding datapath Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

Lucas Saldanha

commit sha 5f43ca4b9bc449b70af3ced13721dee9972d57d7

Fixing test Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in 24 days

push eventlucassaldanha/teku

Anton Nashatyrev

commit sha 45701a4e73fd4f695eaf765090dbf48e6a318350

Remove TODO comment: deposit count shouldn't overflow int (#2677)

view details

Cem Ozer

commit sha 97b5ce37ae5f284ab558030d25bd446438cc6a48

Support reading withdrawal public key from keystore (#2659)

view details

Cem Ozer

commit sha ebad32bd62b94a513d729af95f5d4c930d2f41bd

Use a new constants file for quick acceptance tests (#2674)

view details

Cem Ozer

commit sha 9f48bdb54184102ede7d1c00af6dbd45687f601e

Integrate fork choice tests (#2619)

view details

Adrian Sutton

commit sha d5c28a939bfde85755308c1beeece7ff5f0fc0ec

Find the common ancestor for chain reorgs and use that information to limit the number of epochs we invalidate duties for (#2682)

view details

Adrian Sutton

commit sha 074de9c2401ac43558e491fdf55a701cd7fd375f

Include committee index in log message when aggregation is skipped. (#2689)

view details

Adrian Sutton

commit sha cd23274b21c8e5b38762a1dc3792927f87688547

Avoid performing duplicate tasks when they are rebased (#2690)

view details

Adrian Sutton

commit sha 2882a5be75d14ed4e97dd9ba72f4ef8dba5e638a

We perform full validation on submitted attestations. Remove todo. (#2691)

view details

Adrian Sutton

commit sha a9abcb472cab80cda3652268aec2a03ee8bfc1d7

Ignore env var for --version and --help args. (#2692)

view details

Adrian Sutton

commit sha 316b62866de068af2def44c764639a99c85ebd09

Update discovery. (#2694)

view details

Adrian Sutton

commit sha c6f6d86d1577a6b707e88e8d0ec3ef954d8acc2e

Upgrade RocksDB, sync WAL at shutdown (#2693)

view details

Anton Nashatyrev

commit sha be8d6dce4f82a27b7d18ea356e95bc7db93e64a1

Additional info for exception when Bytes32 behaves inconsistently (#2688)

view details

Anton Nashatyrev

commit sha 43aa9b4689def05c3926c8c831056a2c706f5334

Check gossip message size by ByteBuf.readableBytes() instead of ByteBuf.capacity() (#2696) * Check gossip message size by ByteBuf.readableBytes() instead of ByteBuf.capacity() * Add testcase when buffer capacity is larger then max message size while the readable bytes fit the max size

view details

Paul Harris

commit sha 538c053087f41d08c7d734851254dd6021f4c367

[issue 2601] signingrecord module (#2707) * split out validatorSigningRecord to its own module partially addresses #2601 Signed-off-by: Paul Harris <paul.harris@consensys.net>

view details

mbaxter

commit sha 555b341dc22694efddb8874d3b9aca3795c2d91e

[Issue-2566] Add additional validation when collecting deposits for block proposal (#2703)

view details

Anton Nashatyrev

commit sha e5df3e7315ffcd3119bc9e6b65e205cb39500938

Bump Libp2p version to 0.5.6 (#2710)

view details

Adrian Sutton

commit sha 3d9a206a4bb0075c663663b27d7c61c2164c625c

Remove support for unencrypted keystore generation (#2709)

view details

Usman Saleem

commit sha 9963175d61de56bd09229b244e0e675c365008ca

BLS sign and verify against arbitrary domain separation tag (DST) (#2706) Modify BLS class(es) to allow using an arbitrary DST value to sign (a message) and verify a signature. This is not directly used by the Ethereum standard, however, teku's BLS module is used by other projects which require supporting of sign/verify with a non-ETH DST Signed-off-by: Usman Saleem <usman@usmans.info>

view details

Adrian Sutton

commit sha 4946cc1ce4372d595fd3c3e4f824bb3f3417e129

Update signers to include unicode password normalisation (#2713)

view details

Lucas Saldanha

commit sha ed74c1cb0d0ce9dadb85be6f98b950b013517d0c

Added validator client subcommand

view details

push time in 24 days

Pull request review commentPegaSysEng/teku

Running Validator Node independently

 public TekuConfiguration build() {           "eth1-deposit-contract-address is required if eth1-endpoint is specified.");     } +    // TODO validate config for validator node only

Maybe here we should probably validate that, when running as a validator, the user isn't using some of the bacon node only options (and vice versa?).

lucassaldanha

comment created time in a month

PullRequestReviewEvent

PR opened PegaSysEng/teku

Reviewers
Running Validator Node independently

PR Description

  • Options for starting only a Validator Node (connecting to a Beacon Node).

Fixed Issue(s)

fixes #2450

Documentation

  • [X] I thought about documentation and added the documentation label to this PR if updates are required.
+320 -24

0 comment

13 changed files

pr created time in a month

create barnchlucassaldanha/teku

branch : separate-validator

created branch time in a month

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commenthyperledger/besu

Implemented priv_debugGetStateRoot JSON RPC API.

 void verifyPrivacyGroupContainsEnclavePublicKey(    PrivateTransactionSimulator getTransactionSimulator(); -  Optional<Hash> getBlockHashByBlockNumber(Optional<Long> blockNumber);+  Optional<Hash> getBlockHashByBlockNumber(final Long blockNumber);

Do we need to expose this method in this interface? Isn't it used internally?

I recon we could just find the blockhash from the number inside getStateRootByBlockNumber() method.

mark-terry

comment created time in a month

PullRequestReviewEvent

delete branch lucassaldanha/teku

delete branch : validator-sub-api

delete time in a month

push eventPegaSysEng/teku

Lucas Saldanha

commit sha cd5729c192b1d6bfdf27d43a0f5b84d35856c3e7

Validator client option to subscribe to remote websocket events (#2645) * Validator client option to subscribe to remove websocket events Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in a month

PR merged PegaSysEng/teku

Validator client option to subscribe to remote websocket events

PR Description

  • Refactored AnticorruptionLayer into BeaconChainEventAdapter
    • One implementation uses the EventChannel (as before)
    • A new implementation uses the WebSockets subscription to receive events from the beacon node
  • All the config options and separation of a validator client will be done in a separate PR

Fixed Issue(s)

fixes #2448

Documentation

  • [x] I thought about documentation and added the documentation label to this PR if updates are required.
+364 -67

0 comment

22 changed files

lucassaldanha

pr closed time in a month

issue closedPegaSysEng/teku

Implement Validator subscription to Beacon Node events

Implement a remote version of the antiCorruption layer that read events from the EventChannel and trigger validator actions. We should have an implementation that consumes events from the Beacon Node Remote Validator WebSocket channel.

closed time in a month

lucassaldanha

push eventlucassaldanha/teku

Lucas Saldanha

commit sha 92da04cd43d29d85b2fc812db965a49f3a36829b

PR comments

view details

push time in a month

push eventlucassaldanha/teku

Adrian Sutton

commit sha bd3eeb963e0d8078da8467eb0fad8877fecc003e

Process attestations received in blocks while syncing (#2643)

view details

Cem Ozer

commit sha 93d6ef0a5ef4a617ba4e9fe7c5d32af577c64ec1

Do not add attestations from different forks to produced block (#2632)

view details

Adrian Sutton

commit sha bed34b73015a360ae10a3c4619a78f2623a69c46

Exclude stack trace for most common places RejectedExecutionException is reported. (#2646)

view details

Paul Harris

commit sha 559c1d7b339611bc19ff3905f92d9a411d4ce9c2

split out duties scheduler into separate schedulers (#2636) * split out duties scheduler into separate schedulers - attestation and block production schedulers have different requirements, so splitting them out will allow different processing Incomplete - several regression tests need to be enabled again - further testing on validator nodes other than just minimal will be needed to ensure we haven't degraded processing. partially addresses #2529 Signed-off-by: Paul Harris <paul.harris@consensys.net>

view details

Adrian Sutton

commit sha ed635bbe1b3df52446ebd56b5c9a33864e80563b

Update changelog for 0.12.4 (#2649)

view details

Adrian Sutton

commit sha 4bbd0ac708eb1da8c8a3188001b9fb71fac7b766

0.12.4 release (#2650) Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>

view details

Adrian Sutton

commit sha 3d126dca6e70871cabb4232c946a7936e0cf8867

Prepare for version 0.12.5-SNAPSHOT (#2651) Signed-off-by: Adrian Sutton <adrian.sutton@consensys.net>

view details

Lucas Saldanha

commit sha 0338c92373223475e27d83e0edee8d721746559e

Validator client option to subscribe to remove websocket events Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in a month

PR opened PegaSysEng/teku

Validator client option to subscribe to remote websocket events

PR Description

  • Refactored AnticorruptionLayer into BeaconChainEventAdapter
    • One implementation uses the EventChannel (as before)
    • A new implementation uses the WebSockets subscription to receive events from the beacon node
  • All the config options and separation of a validator client will be done in a separate PR

Fixed Issue(s)

fixes #2448

Documentation

  • [x] I thought about documentation and added the documentation label to this PR if updates are required.
+234 -67

0 comment

20 changed files

pr created time in a month

create barnchlucassaldanha/teku

branch : validator-sub-api

created branch time in a month

PullRequestReviewEvent

delete branch lucassaldanha/teku

delete branch : fix-remote-validator-test

delete time in a month

push eventPegaSysEng/teku

Lucas Saldanha

commit sha 26b8cd95f8eb0946bbed89cd5d23363fb7748ba6

Fix RemoteValidatorApiHandlerTest unit test (#2634) Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in a month

PR merged PegaSysEng/teku

Fix RemoteValidatorApiHandlerTest unit test

PR Description

  • Fix intermittent unit test

Documentation

  • [x] I thought about documentation and added the documentation label to this PR if updates are required.
+10 -3

0 comment

1 changed file

lucassaldanha

pr closed time in a month

PR opened PegaSysEng/teku

Reviewers
Fix RemoteValidatorApiHandlerTest unit test

PR Description

  • Fix intermittent unit test

Documentation

  • [x] I thought about documentation and added the documentation label to this PR if updates are required.
+10 -3

0 comment

1 changed file

pr created time in a month

create barnchlucassaldanha/teku

branch : fix-remote-validator-test

created branch time in a month

startedHabrador/Unity-Programming-Patterns

started time in a month

PullRequestReviewEvent

delete branch lucassaldanha/teku

delete branch : add-remote-validator-service

delete time in a month

PR merged PegaSysEng/teku

Added remote validator service to ServiceController

PR Description

  • Added Remote Validator service to ServiceController.
  • If BeaconNode is started with option --Xremote-validator-api-enabled, we start a RemoteValidatorService and not a ValidatorClientService instance.
  • Reduced visibility in some internal remove validator classes

Documentation

  • [x] I thought about documentation and added the documentation label to this PR if updates are required.
+26 -21

0 comment

8 changed files

lucassaldanha

pr closed time in a month

push eventPegaSysEng/teku

Lucas Saldanha

commit sha 4c19b9b0742b2593bb23cfb2b061369f016346b7

Added remote validator service to ServiceController (#2627) Added remote validator service to ServiceController (#2627) Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in a month

PR opened PegaSysEng/teku

Reviewers
Added remote validator service to ServiceController

PR Description

  • Added Remote Validator service to ServiceController.
  • If BeaconNode is started with option --Xremote-validator-api-enabled, we start a RemoteValidatorService and not a ValidatorClientService instance.

Documentation

  • [x] I thought about documentation and added the documentation label to this PR if updates are required.
+26 -21

0 comment

8 changed files

pr created time in a month

create barnchlucassaldanha/teku

branch : add-remote-validator-service

created branch time in a month

PullRequestReviewEvent

Pull request review commenthyperledger/besu

Private Log Filters track enclavePublicKey and are removed when user removed from group

 public Bytes compute(final Bytes input, final MessageFrame messageFrame) {     }      if (privateTransaction.isGroupRemovalTransaction()) {+      // get first participant parameter - there will be only one for removal transaction+      final String removedParticipant = getParticipantsFromParameter(privateTransaction.getPayload()).get(0);+       final PrivateTransactionEvent removalEvent =           new PrivateTransactionEvent(               privateTransaction.getPrivacyGroupId().get().toBase64String(),-              privateTransaction.getPrivateFrom().toBase64String());+              removedParticipant);       privateTransactionEventObservers.forEach(           sub -> sub.onPrivateTransactionProcessed(removalEvent));

We could move this to a private method SendRemovedParticipantEvent() to try to keep the compute() method a bit cleaner (I know it is already kinda messy, but let's try to not make it worse) :)

macfarla

comment created time in a month

PullRequestReviewEvent

delete branch lucassaldanha/teku

delete branch : validator-using-apis

delete time in a month

push eventPegaSysEng/teku

Lucas Saldanha

commit sha de31bf679c9852fa3b914894a2bf5064724caef2

#2449 Validator API REST client (#2588) * #2449 Validator API REST client Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in a month

PR merged PegaSysEng/teku

#2449 Validator API REST client

PR Description

  • Implemented a client to consume the Validator API

Fixed Issue(s)

fixes #2449

Documentation

  • [X] I thought about documentation and added the documentation label to this PR if updates are required.
+1572 -15

0 comment

16 changed files

lucassaldanha

pr closed time in a month

issue closedPegaSysEng/teku

Add REST API as an option for msgs from the Validator to the Beacon Node

The current implementation of the ValidatorApiChannel relies on the EventChannel to read/post information to the Beacon Node. We need an implementation that uses the REST API provided by the Beacon Node instead.

closed time in a month

lucassaldanha

push eventlucassaldanha/teku

Adrian Sutton

commit sha a74dac8d2950848eaa3bbcdeb842fa8b8c02e266

Use Guava Cache when creating `LimitedMap` rather than `LinkedHashMap`. (#2613) Concurrent access safe by default (and much more efficiently than using synchronizedMap) and will fix an issue where LinkedHashMap was unexpectedly exceeding it's expected maximum size. Only least recently used eviction is supported. This was used in all but one place which never accessed individual items anyway so is unaffected by the change.

view details

Lucas Saldanha

commit sha f62c491dbe7e877bed1172ae465a548430b6c7ff

Merge branch 'master' into validator-using-apis

view details

push time in a month

push eventlucassaldanha/teku

Lucas Saldanha

commit sha b39ec98c715d25c669aa09a19a5116c2e9493e17

Fix unit test Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in a month

push eventlucassaldanha/teku

Adrian Sutton

commit sha 0c78ab25dfcd2dc3494632e075c2683d3a37f3f2

Log a message when states are being regenerated. (#2591)

view details

Adrian Sutton

commit sha e6f614d5dc534193b386bd484d2f3b59d8df80ce

Add state regeneration queue to limit concurrent regenerations (#2589)

view details

Paul Harris

commit sha 25bf7117cdabe4b73227b253343d20842abe0e6c

Removed secIO support (#2595) fixes #2574 Signed-off-by: Paul Harris <paul.harris@consensys.net>

view details

Adrian Sutton

commit sha da55662689a80260f8838c62841d7ed1530971a5

Be more lenient before deciding a peer is excessively throttling requests (#2590) Co-authored-by: Meredith Baxter <meredith.baxter@consensys.net>

view details

Adrian Sutton

commit sha 3ad30d83e54b37ba4e216957b39675157270bb8d

Add counters to StateGenerationQueue to track how many requests can be deduplicated or rebased to reduce the scope. (#2597)

view details

Adrian Sutton

commit sha 8e98ffa3525e5ae5b7a1a85b574a08ad2ace57d5

Check if a checkpoint state is available from cache before regenerating. (#2599)

view details

Adrian Sutton

commit sha afa6cf96339532a7178348f3225949ab4a27c307

Ensure votes map is only accessed within the read/write lock. (#2603)

view details

Adrian Sutton

commit sha 605d4062df7dcff3f6c6b1c466cb68def1cf7b93

Add tests for SafeFuture.alwaysRun (#2605)

view details

Adrian Sutton

commit sha 39bfecad0bff96e5ab31aaca970929f45b61b94d

Fix race condition when committing changes to fork choice votes (#2602)

view details

Cem Ozer

commit sha d912c3c0bec3bf33629309d9fcc53060ee2eb2bf

Update to latest Web3j version (#2607)

view details

Cem Ozer

commit sha 6e36dccf9b6113ff41c4c449a419fdc6f9b64bcb

Use cache to get block and state (#2610) * Use the cache for getting block and state * Run spotless

view details

Adrian Sutton

commit sha c434b8f7118427bf1fc50a52e095ead7877b2460

Use multiple threads to service ValidatorApiChannel (#2572) Use an AsyncRunner for multithreaded EventChannels instead of a fixed number of threads.

view details

Lucas Saldanha

commit sha eab7c59f9c04992eeeba5adbe8f488cae660b7c7

PR comments Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

Lucas Saldanha

commit sha 3c41b10f7cf189d798571e1da51f6423c62f3fe1

Merge branch 'master' into validator-using-apis Signed-off-by: Lucas Saldanha <lucas.saldanha@consensys.net>

view details

push time in a month

Pull request review commentPegaSysEng/teku

#2449 Validator API REST client

+/*+ * Copyright 2020 ConsenSys AG.+ *+ * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with+ * the License. You may obtain a copy of the License at+ *+ * http://www.apache.org/licenses/LICENSE-2.0+ *+ * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the+ * specific language governing permissions and limitations under the License.+ */++package tech.pegasys.teku.validator.remote.apiclient;++import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.GET_AGGREGATE;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.GET_DUTIES;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.GET_FORK;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.GET_UNSIGNED_ATTESTATION;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.GET_UNSIGNED_BLOCK;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.SEND_SIGNED_AGGREGATE_AND_PROOF;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.SEND_SIGNED_ATTESTATION;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.SEND_SIGNED_BLOCK;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.SUBSCRIBE_TO_COMMITTEE_FOR_AGGREGATION;+import static tech.pegasys.teku.validator.remote.apiclient.ValidatorApiMethod.SUBSCRIBE_TO_PERSISTENT_SUBNETS;++import com.fasterxml.jackson.core.JsonProcessingException;+import com.google.common.annotations.VisibleForTesting;+import java.io.IOException;+import java.util.Arrays;+import java.util.Collections;+import java.util.HashMap;+import java.util.List;+import java.util.Map;+import java.util.Optional;+import java.util.Set;+import okhttp3.HttpUrl;+import okhttp3.MediaType;+import okhttp3.OkHttpClient;+import okhttp3.Request;+import okhttp3.RequestBody;+import okhttp3.Response;+import org.apache.commons.lang3.StringUtils;+import org.apache.logging.log4j.LogManager;+import org.apache.logging.log4j.Logger;+import org.apache.tuweni.bytes.Bytes32;+import tech.pegasys.teku.api.request.SubscribeToBeaconCommitteeRequest;+import tech.pegasys.teku.api.response.GetForkResponse;+import tech.pegasys.teku.api.schema.Attestation;+import tech.pegasys.teku.api.schema.BLSSignature;+import tech.pegasys.teku.api.schema.BeaconBlock;+import tech.pegasys.teku.api.schema.SignedAggregateAndProof;+import tech.pegasys.teku.api.schema.SignedBeaconBlock;+import tech.pegasys.teku.api.schema.SubnetSubscription;+import tech.pegasys.teku.api.schema.ValidatorDuties;+import tech.pegasys.teku.api.schema.ValidatorDutiesRequest;+import tech.pegasys.teku.infrastructure.unsigned.UInt64;+import tech.pegasys.teku.provider.JsonProvider;++public class OkHttpValidatorRestApiClient implements ValidatorRestApiClient {++  private static final Logger LOG = LogManager.getLogger();++  private static final MediaType APPLICATION_JSON =+      MediaType.parse("application/json; charset=utf-8");+  private static final Map<String, String> EMPTY_QUERY_PARAMS = Collections.emptyMap();++  private final JsonProvider jsonProvider = new JsonProvider();+  private final OkHttpClient httpClient;+  private final HttpUrl baseEndpoint;++  public OkHttpValidatorRestApiClient(final String baseEndpoint) {+    this(HttpUrl.parse(baseEndpoint), new OkHttpClient());+  }++  @VisibleForTesting+  OkHttpValidatorRestApiClient(final HttpUrl baseEndpoint, final OkHttpClient okHttpClient) {+    this.baseEndpoint = baseEndpoint;+    this.httpClient = okHttpClient;+  }++  @Override+  public Optional<GetForkResponse> getFork() {+    return get(GET_FORK, EMPTY_QUERY_PARAMS, GetForkResponse.class);+  }++  @Override+  @SuppressWarnings("unchecked")+  public List<ValidatorDuties> getDuties(final ValidatorDutiesRequest request) {+    return post(GET_DUTIES, request, ValidatorDuties[].class)+        .map(Arrays::asList)+        .orElse(Collections.EMPTY_LIST);+  }++  @Override+  public Optional<BeaconBlock> createUnsignedBlock(+      final UInt64 slot, final BLSSignature randaoReveal, final Optional<Bytes32> graffiti) {+    final Map<String, String> queryParams = new HashMap<>();+    queryParams.put("slot", encodeQueryParam(slot));+    queryParams.put("randao_reveal", encodeQueryParam(randaoReveal));+    graffiti.ifPresent(bytes32 -> queryParams.put("graffiti", encodeQueryParam(bytes32)));++    return get(GET_UNSIGNED_BLOCK, queryParams, BeaconBlock.class);+  }++  @Override+  public void sendSignedBlock(final SignedBeaconBlock beaconBlock) {+    post(SEND_SIGNED_BLOCK, beaconBlock, null);+  }++  @Override+  public Optional<Attestation> createUnsignedAttestation(+      final UInt64 slot, final int committeeIndex) {+    final Map<String, String> queryParams = new HashMap<>();+    queryParams.put("slot", encodeQueryParam(slot));+    queryParams.put("committee_index", String.valueOf(committeeIndex));++    return get(GET_UNSIGNED_ATTESTATION, queryParams, Attestation.class);+  }++  @Override+  public void sendSignedAttestation(final Attestation attestation) {+    post(SEND_SIGNED_ATTESTATION, attestation, null);+  }++  @Override+  public Optional<Attestation> createAggregate(final Bytes32 attestationHashTreeRoot) {+    final Map<String, String> queryParams = new HashMap<>();+    queryParams.put("slot", encodeQueryParam(UInt64.ZERO));+    queryParams.put("attestation_data_root", encodeQueryParam(attestationHashTreeRoot));++    return get(GET_AGGREGATE, queryParams, Attestation.class);+  }++  @Override+  public void sendAggregateAndProof(final SignedAggregateAndProof signedAggregateAndProof) {+    post(SEND_SIGNED_AGGREGATE_AND_PROOF, signedAggregateAndProof, null);+  }++  @Override+  public void subscribeToBeaconCommitteeForAggregation(+      final int committeeIndex, final UInt64 aggregationSlot) {+    final SubscribeToBeaconCommitteeRequest request =+        new SubscribeToBeaconCommitteeRequest(committeeIndex, aggregationSlot);+    post(SUBSCRIBE_TO_COMMITTEE_FOR_AGGREGATION, request, null);+  }++  @Override+  public void subscribeToPersistentSubnets(final Set<SubnetSubscription> subnetSubscriptions) {+    post(SUBSCRIBE_TO_PERSISTENT_SUBNETS, subnetSubscriptions, null);+  }++  public <T> Optional<T> get(+      final ValidatorApiMethod apiMethod,+      final Map<String, String> queryParams,+      final Class<T> responseClass) {+    final HttpUrl.Builder httpUrlBuilder = urlBuilder(apiMethod);+    if (queryParams != null && !queryParams.isEmpty()) {+      queryParams.forEach(httpUrlBuilder::addQueryParameter);+    }++    final Request request = new Request.Builder().url(httpUrlBuilder.build()).build();+    return executeCall(request, responseClass);+  }++  private <T> Optional<T> post(+      final ValidatorApiMethod apiMethod,+      final Object requestBodyObj,+      final Class<T> responseClass) {+    final String requestBody;+    try {+      requestBody = jsonProvider.objectToJSON(requestBodyObj);+    } catch (JsonProcessingException e) {+      throw new RuntimeException(e);+    }++    final Request request =+        new Request.Builder()+            .url(urlBuilder(apiMethod).build())+            .post(RequestBody.create(requestBody, APPLICATION_JSON))+            .build();++    return executeCall(request, responseClass);+  }++  private HttpUrl.Builder urlBuilder(final ValidatorApiMethod apiMethod) {+    return new HttpUrl.Builder()+        .scheme(baseEndpoint.scheme())+        .host(baseEndpoint.host())+        .port(baseEndpoint.port())+        .encodedPath(apiMethod.getPath());+  }++  private <T> Optional<T> executeCall(final Request request, final Class<T> responseClass) {+    try (final Response response = httpClient.newCall(request).execute()) {+      LOG.trace("{} {} {}", request.method(), request.url(), response.code());++      switch (response.code()) {+        case 200:+          {+            final String responseBody = response.body().string();+            if (responseClass != null) {+              final T responseObj = jsonProvider.jsonToObject(responseBody, responseClass);+              return Optional.of(responseObj);+            } else {+              return Optional.empty();+            }+          }+        case 202:+        case 204:+        case 404:+        case 503:+          {+            return Optional.empty();+          }+        case 400:+          {+            LOG.warn("Invalid params response from Beacon Node API - {}", response.body().string());+            return Optional.empty();+          }+        default:+          {+            final String responseBody = response.body().string();+            LOG.error(+                "Unexpected error calling Beacon Node API (status = {}, response = {})",+                response.code(),+                responseBody);+            throw new RuntimeException(+                "Unexpected response from Beacon Node API (status = "+                    + response.code()+                    + ", response = "+                    + responseBody+                    + ")");+          }+      }+    } catch (IOException e) {+      LOG.error("Error communicating with Beacon Node API", e);+      throw new RuntimeException(e);+    }+  }++  private String encodeQueryParam(final Object value) {+    try {+      return removeQuotesIfPresent(jsonProvider.objectToJSON(value));+    } catch (JsonProcessingException e) {+      throw new RuntimeException(+          "Can't encode param of type " + value.getClass().getSimpleName(), e);+    }+  }++  private String removeQuotesIfPresent(final String value) {+    StringUtils.strip(value, "\"");

Ah, I was playing around with it :)

lucassaldanha

comment created time in a month

PullRequestReviewEvent
more