profile
viewpoint
Dandelion Mané decentralion @SourceCred Seattle building @sourcecred, a reputation protocol for open collaboration they/them

decentralion/chartographer 46

Super simple charts via Plottable and D3

decentralion/bacteria-sim 4

An artistic project to simulate natural selection in the browser

decentralion/Bhattacharrya 4

Implementing the recursive Bhattacharrya kernel

decentralion/abalone 2

an Abalone AI programming tournament

decentralion/Animal-Game 2

Animal Game, implemented in C with serialization

decentralion/Bitcoin-Viz 2

Visualize flows of bitcoins

decentralion/abalone-engine 1

Engine for an abalone implementation for an AI programming competition

decentralion/bcparse 1

a bitcoin parser in java

decentralion/C-Modules 1

Useful reusable modules written in C

PullRequestReviewEvent

push eventsourcecred/cred

Dandelion Mané

commit sha e1d5c3f9324794a60cebeef448d800b8d8ed46f9

Record grain transactions Grain transfer from LB to Joie Grain sale from Joie to SourceCred

view details

push time in 2 days

pull request commentShenaniganDApp/scoreboard

Upgrade SourceCred to beta-16

(Note: I can't test this b.c. I don't have the Discord key. This should work, though)

decentralion

comment created time in 2 days

pull request commentShenaniganDApp/scoreboard

Upgrade SourceCred to beta-16

cc @youngkidwarrior

decentralion

comment created time in 2 days

PR opened ShenaniganDApp/scoreboard

Upgrade SourceCred to beta-16

Fixes the build break induced by #2

+87 -54

0 comment

2 changed files

pr created time in 2 days

create barnchdecentralion/scoreboard

branch : upgrade-sc

created branch time in 2 days

pull request commentShenaniganDApp/scoreboard

Flow Cred to SourceCred as a dependency

Oops, sorry, the version needs to be ugpraded.

decentralion

comment created time in 2 days

push eventsourcecred/cred

Dandelion Mané

commit sha 39864f770789522754f037aa0cf1432ce5c1d47d

Record transfers: From SC to Protocol From YoungKidWarrior to SC

view details

push time in 3 days

pull request commentShenaniganDApp/scoreboard

Flow Cred to SourceCred as a dependency

ping @youngkidwarrior

decentralion

comment created time in 3 days

PullRequestReviewEvent

PR closed sourcecred/makerdao-cred

Reviewers
Scheduled grain distribution for week ending September 20th, 2020

This PR was auto-generated on 20-09-2020 to add the latest grain distribution to our instance.

yarn run v1.22.5 $ sourcecred grain Distributed 2,500g to 26 identities in 1 distributions Done in 1.79s.

+35 -0

1 comment

1 changed file

github-actions[bot]

pr closed time in 3 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

push eventsourcecred/sourcecred

Dandelion Mané

commit sha 359d656e172b2d5b9d3fd016f4dd8a90da5a952f

Update grain helptext

view details

Dandelion Mané

commit sha 43714fb1ea4dece69315182ee9bf97d1d284b543

document the no-load argument to cli/go

view details

Dandelion Mané

commit sha 7750292ed47c7900862e2cfd5a0226a35b92d2d9

document that serve automatically runs site

view details

push time in 3 days

pull request commentsourcecred/sourcecred

Merge graph and score CLI commands

Closing b.c. we decided to go in a different direction.

topocount

comment created time in 4 days

PR closed sourcecred/sourcecred

Reviewers
Merge graph and score CLI commands

In order to simplify workflows for users and devs, these commands were merged together. setup logic was deduped (at least enough to make flow happy)

test plan: yarn load && yarn score in an up-to-date instance. Relevant unit tests have been migrated

+109 -151

1 comment

7 changed files

topocount

pr closed time in 4 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

credrank: create epoch accumulator nodes

 const FIBRATION_EDGE = EdgeAddress.fromParts([ ]); const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT"); const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");-const EPOCH_RADIATION = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_RADIATION");+const USER_EPOCH_RADIATION = EdgeAddress.append(+  FIBRATION_EDGE,+  "USER_EPOCH_RADIATION"+);

Let's remove?

wchargin

comment created time in 4 days

Pull request review commentsourcecred/sourcecred

credrank: create epoch accumulator nodes

 const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED"); const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING  // Node address prefix for epoch nodes.-const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");+const USER_EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "USER_EPOCH");  export type EpochNodeAddress = {|-  +type: "EPOCH_NODE",+  +type: "USER_EPOCH",   +owner: NodeAddressT,   +epochStart: TimestampMs, |};  export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {

userEpochNodeAddressToRaw for consistency?

wchargin

comment created time in 4 days

Pull request review commentsourcecred/sourcecred

credrank: create epoch accumulator nodes

 const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED"); const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING  // Node address prefix for epoch nodes.-const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");+const USER_EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "USER_EPOCH");  export type EpochNodeAddress = {|

UserEpochNodeAddress for consistency?

wchargin

comment created time in 4 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventdecentralion/random-playground

Dandelion Mané

commit sha 1623a085b232dbf5247b3ebd2576fb14bcb647aa

Create README.md

view details

push time in 5 days

create barnchdecentralion/random-playground

branch : master

created branch time in 5 days

pull request commentsourcecred/cred

Scheduled grain distribution for week ending September 21th, 2020

Wound up doing this manually, b.c. I wanted to process an opt-in.

github-actions[bot]

comment created time in 5 days

PR closed sourcecred/cred

Reviewers
Scheduled grain distribution for week ending September 21th, 2020

This PR was auto-generated on 21-09-2020 to add the latest grain distribution to our instance.

yarn run v1.22.5 $ sourcecred grain Distributed 25,000g to 40 identities in 1 distributions Done in 1.41s.

+25 -0

1 comment

1 changed file

github-actions[bot]

pr closed time in 5 days

push eventsourcecred/cred

Dandelion Mané

commit sha ed3965f91d318b7374b89fd42ff32ba36c9e8cd6

Automatic ledger identity additions

view details

Dandelion Mané

commit sha 463949540779805d0f5f2027be049627094b649d

activate coopahtroopa

view details

Dandelion Mané

commit sha 822e9cb42e4a54379539c736fa727a6cef2efd6a

Grain distribution

view details

push time in 5 days

PR opened ShenaniganDApp/scoreboard

Flow Cred to SourceCred as a dependency

This adds a dependencies.json file, with SourceCred as a dependency, set to auto-activate (i.e. receive PRTCL) and with a weight of 0.05.

I set the start date for yesterday, b.c. it seems like that's when SC started being used to distribute PRTCL.

Thanks! :)

+12 -0

0 comment

1 changed file

pr created time in 5 days

create barnchdecentralion/scoreboard

branch : cred-to-sc

created branch time in 5 days

PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

Add no-load flag to go CLI command

 function die(std, message) {   return 1; } +const noLoadArg = "--no-load";+ const goCommand: Command = async (args, std) => {-  if (args.length !== 0) {-    return die(std, "usage: sourcecred go");-  }   const commandSequence = [     {name: "load", command: load},     {name: "graph", command: graph},     {name: "score", command: score},   ];++  if (args.length === 1 && args[0] === noLoadArg) {+    commandSequence.shift();

IMO this is a little bit cute and would generally prefer something more explicit, but this command is so trivial that I think its fine.

topocount

comment created time in 7 days

Pull request review commentsourcecred/sourcecred

Add no-load flag to go CLI command

 function die(std, message) {   return 1; } +const noLoadArg = "--no-load";

we generally CAPS_CASE args like this

topocount

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

Call site CLI command inside serve

 // @flow +import type {$Response as ExpressResponse} from "express";+ import type {Command} from "./command"; import {loadInstanceConfig} from "./common";-import type {$Response as ExpressResponse} from "express";+import site from "./site";

suggest importing as siteCommand for consistency.

topocount

comment created time in 8 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *seed-in* / "redistribution" / "radiation" edges from nodes to+ *     the seed node;+ *   - *seed-out* / "minting" edges from the seed node to cred-minting+ *     nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).

Right.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *seed-in* / "redistribution" / "radiation" edges from nodes to+ *     the seed node;+ *   - *seed-out* / "minting" edges from the seed node to cred-minting+ *     nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {type NodeWeight} from "./weights";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TimestampMs = number;+export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +weight: NodeWeight,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.fromParts(["sourcecred", "core", "SEED"]);+const SEED_DESCRIPTION = "\u{1f331}"; // SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.fromParts(["sourcecred", "core", "EPOCH"]);++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+// Prefixes for seed edges.+const SEED_IN = EdgeAddress.fromParts(["sourcecred", "core", "SEED_IN"]);+const SEED_OUT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_OUT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(node, prefix)` for+  // some element `prefix` of `what`.+  +what: $ReadOnlyArray<NodeAddressT>,+  // Transition probability for payout edges from epoch nodes to their+  // owners.+  +beta: TransitionProbability,+  // Transition probability for webbing edges from an epoch node to the+  // next epoch node for the same owner.+  +gammaForward: TransitionProbability,+  +gammaBackward: TransitionProbability,+|};+export type SeedOptions = {|+  +alpha: TransitionProbability,+|};++const COMPAT_INFO = {type: "sourcecred/markovProcessGraph", version: "0.1.0"};++export type MarkovProcessGraphJSON = Compatible<{|+  +nodes: {|+[NodeAddressT]: MarkovNode|},+  +edges: {|+[MarkovEdgeAddressT]: MarkovEdge|},+  +scoringAddresses: $ReadOnlyArray<NodeAddressT>,+|}>;++export class MarkovProcessGraph {+  _nodes: Map<NodeAddressT, MarkovNode>;+  _edges: Map<MarkovEdgeAddressT, MarkovEdge>;+  _scoringAddresses: Set<NodeAddressT>;++  constructor(+    nodes: Map<NodeAddressT, MarkovNode>,+    edges: Map<MarkovEdgeAddressT, MarkovEdge>,+    scoringAddresses: Set<NodeAddressT>+  ) {+    this._nodes = nodes;+    this._edges = edges;+    this._scoringAddresses = scoringAddresses;+  }++  static new(+    wg: WeightedGraphT,+    fibration: FibrationOptions,+    seed: SeedOptions+  ) {+    const _nodes = new Map();+    const _edges = new Map();+    const _scoringAddresses = _findScoringAddresses(wg.graph, fibration.what);++    const epochTransitionRemainder = (() => {+      const {beta, gammaForward, gammaBackward} = fibration;+      if (beta < 0 || gammaForward < 0 || gammaBackward < 0) {+        throw new Error(+          "Negative transition probability: " ++            [beta, gammaForward, gammaBackward].join(" or ")+        );+      }+      const result = 1 - (beta + gammaForward + gammaBackward);+      if (result < 0) {+        throw new Error("Overlarge transition probability: " + (1 - result));+      }+      return result;+    })();++    const timeBoundaries = (() => {+      const edgeTimestamps = Array.from(+        wg.graph.edges({showDangling: false})+      ).map((x) => x.timestampMs);+      const start = min(edgeTimestamps);+      const end = max(edgeTimestamps);+      const boundaries = weekIntervals(start, end).map((x) => x.startTimeMs);+      return [-Infinity, ...boundaries, Infinity];+    })();++    const addNode = (node: MarkovNode) => {+      if (_nodes.has(node.address)) {+        throw new Error("Node conflict: " + node.address);+      }+      _nodes.set(node.address, node);+    };+    const addEdge = (edge: MarkovEdge) => {+      const mae = rawEdgeAddress(edge);+      if (_edges.has(mae)) {+        throw new Error("Edge conflict: " + mae);+      }+      _edges.set(mae, edge);+    };++    // Add seed node+    addNode({+      address: SEED_ADDRESS,+      description: SEED_DESCRIPTION,+      weight: 0,+    });++    // Add graph nodes+    const nwe = nodeWeightEvaluator(wg.weights);+    for (const node of wg.graph.nodes()) {+      const weight = nwe(node.address);+      if (weight < 0) {+        const name = NodeAddress.toString(node.address);+        throw new Error(`Negative node weight for ${name}: ${weight}`);+      }+      addNode({+        address: node.address,+        description: node.description,+        weight,+      });+    }++    // Add epoch nodes, payout edges, and epoch webbing+    for (const scoringAddress of _scoringAddresses) {+      let lastBoundary = null;+      for (const boundary of timeBoundaries) {+        const thisEpoch = epochNodeAddressToRaw({+          type: "EPOCH_NODE",+          owner: scoringAddress,+          epochStart: boundary,+        });+        addNode({+          address: thisEpoch,+          description: `Epoch starting ${boundary} ms past epoch`,+          weight: 0,+        });+        addEdge({+          address: EdgeAddress.append(+            EPOCH_PAYOUT,+            String(boundary),+            ...NodeAddress.toParts(scoringAddress)+          ),+          reversed: false,+          src: thisEpoch,+          dst: scoringAddress,+          transitionProbability: fibration.beta,+        });+        if (lastBoundary != null) {+          const lastEpoch = epochNodeAddressToRaw({+            type: "EPOCH_NODE",+            owner: scoringAddress,+            epochStart: lastBoundary,+          });+          const webAddress = EdgeAddress.append(+            EPOCH_WEBBING,+            String(boundary),+            ...NodeAddress.toParts(scoringAddress)+          );+          addEdge({+            address: webAddress,+            reversed: false,+            src: lastEpoch,+            dst: thisEpoch,+            transitionProbability: fibration.gammaForward,+          });+          addEdge({+            address: webAddress,+            reversed: true,+            src: thisEpoch,+            dst: lastEpoch,+            transitionProbability: fibration.gammaBackward,+          });+        }+        lastBoundary = boundary;+      }+    }++    // Add radiation (seed-in) edges+    for (const node of wg.graph.nodes()) {+      addEdge({+        address: EdgeAddress.append(+          SEED_IN,+          ...NodeAddress.toParts(node.address)+        ),+        reversed: false,+        src: node.address,+        dst: SEED_ADDRESS,+        transitionProbability: seed.alpha,+      });+    }++    // Add minting (seed-out) edges+    {+      let totalNodeWeight = 0.0;+      const positiveNodeWeights: Map<NodeAddressT, number> = new Map();+      for (const {address, weight} of _nodes.values()) {+        if (weight > 0) {+          totalNodeWeight += weight;+          positiveNodeWeights.set(address, weight);+        }+      }+      if (!(totalNodeWeight > 0)) {+        throw new Error("No outflow from seed; add cred-minting nodes");+      }+      for (const [address, weight] of positiveNodeWeights) {+        addEdge({+          address: EdgeAddress.append(+            SEED_OUT,+            ...NodeAddress.toParts(address)+          ),+          reversed: false,+          src: SEED_ADDRESS,+          dst: address,+          transitionProbability: weight / totalNodeWeight,+        });+      }+    }++    /**+     * Find an epoch node, or just the original node if it's not a+     * scoring address.+     */+    const rewriteEpochNode = (+      address: NodeAddressT,+      edgeTimestampMs: number+    ): NodeAddressT => {+      if (!_scoringAddresses.has(address)) {+        return address;+      }+      const epochEndIndex = sortedIndex(timeBoundaries, edgeTimestampMs);+      const epochStartIndex = epochEndIndex - 1;+      const epochTimestampMs = timeBoundaries[epochStartIndex];+      return epochNodeAddressToRaw({+        type: "EPOCH_NODE",+        owner: address,+        epochStart: epochTimestampMs,+      });+    };++    // Add graph edges. First, split by direction.+    type _UnidirectionalGraphEdge = {|+      +address: EdgeAddressT,+      +reversed: boolean,+      +src: NodeAddressT,+      +dst: NodeAddressT,+      +timestamp: TimestampMs,+      +weight: number,+    |};+    const unidirectionalGraphEdges = function* (): Iterator<_UnidirectionalGraphEdge> {+      const ewe = edgeWeightEvaluator(wg.weights);+      for (const edge of (function* () {+        for (const edge of wg.graph.edges({showDangling: false})) {+          const weight = ewe(edge.address);+          yield {+            address: edge.address,+            reversed: false,+            src: edge.src,+            dst: edge.dst,+            timestamp: edge.timestampMs,+            weight: weight.forwards,+          };+          yield {+            address: edge.address,+            reversed: true,+            src: edge.dst,+            dst: edge.src,+            timestamp: edge.timestampMs,+            weight: weight.backwards,+          };+        }+      })()) {+        if (edge.weight > 0) {+          yield edge;+        }+      }+    };++    const srcNodes: Map<+      NodeAddressT /* domain: (nodes with out-edges) U (epoch nodes) */,

in retrospect, this seems a little pedantic, but still correct, so maybe worth following up on.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import type {TimestampMs} from "../util/timestamp";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +mint: number,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++const CORE_NODE_PREFIX = NodeAddress.fromParts(["sourcecred", "core"]);++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED");+const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+const EPOCH_RADIATION = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_RADIATION");++// Prefixes for seed edges.+const CONTRIBUTION_RADIATION = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "CONTRIBUTION_RADIATION",+]);+const SEED_MINT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_MINT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(a, prefix)` for some+  // element `prefix` of `what`.+  +what: $ReadOnlyArray<NodeAddressT>,+  // Transition probability for payout edges from epoch nodes to their+  // owners.+  +beta: TransitionProbability,+  // Transition probability for webbing edges from an epoch node to the+  // next epoch node for the same owner.+  +gammaForward: TransitionProbability,+  +gammaBackward: TransitionProbability,+|};+export type SeedOptions = {|+  +alpha: TransitionProbability,+|};++const COMPAT_INFO = {type: "sourcecred/markovProcessGraph", version: "0.1.0"};++export type MarkovProcessGraphJSON = Compatible<{|+  +nodes: {|+[NodeAddressT]: MarkovNode|},+  +edges: {|+[MarkovEdgeAddressT]: MarkovEdge|},+  +scoringAddresses: $ReadOnlyArray<NodeAddressT>,+|}>;++export class MarkovProcessGraph {+  _nodes: Map<NodeAddressT, MarkovNode>;+  _edges: Map<MarkovEdgeAddressT, MarkovEdge>;+  _scoringAddresses: Set<NodeAddressT>;++  constructor(+    nodes: Map<NodeAddressT, MarkovNode>,+    edges: Map<MarkovEdgeAddressT, MarkovEdge>,+    scoringAddresses: Set<NodeAddressT>+  ) {+    this._nodes = nodes;+    this._edges = edges;+    this._scoringAddresses = scoringAddresses;+  }++  static new(+    wg: WeightedGraphT,+    fibration: FibrationOptions,+    seed: SeedOptions+  ) {+    const _nodes = new Map();+    const _edges = new Map();+    const _scoringAddresses = _findScoringAddresses(wg.graph, fibration.what);++    // _nodeOutMasses[a] = sum(e.pr for e in edges if e.src == a)+    // Used for computing remainder-to-seed edges.+    const _nodeOutMasses = new Map();++    const epochTransitionRemainder = (() => {+      const {alpha} = seed;+      const {beta, gammaForward, gammaBackward} = fibration;+      if (beta < 0 || gammaForward < 0 || gammaBackward < 0) {+        throw new Error(+          "Negative transition probability: " ++            [beta, gammaForward, gammaBackward].join(" or ")+        );+      }+      const result = 1 - (alpha + beta + gammaForward + gammaBackward);+      if (result < 0) {+        throw new Error("Overlarge transition probability: " + (1 - result));+      }+      return result;+    })();++    const timeBoundaries = (() => {+      const edgeTimestamps = Array.from(+        wg.graph.edges({showDangling: false})+      ).map((x) => x.timestampMs);+      const start = min(edgeTimestamps);+      const end = max(edgeTimestamps);+      const boundaries = weekIntervals(start, end).map((x) => x.startTimeMs);+      return [-Infinity, ...boundaries, Infinity];+    })();++    const addNode = (node: MarkovNode) => {+      if (_nodes.has(node.address)) {+        throw new Error("Node conflict: " + node.address);+      }+      _nodes.set(node.address, node);+    };+    const addEdge = (edge: MarkovEdge) => {+      const mae = rawEdgeAddress(edge);+      if (_edges.has(mae)) {+        throw new Error("Edge conflict: " + mae);+      }+      const pr = edge.transitionProbability;+      if (pr < 0 || pr > 1) {+        const name = MarkovEdgeAddress.toString(mae);+        throw new Error(`Invalid transition probability for ${name}: ${pr}`);+      }+      _edges.set(mae, edge);+      _nodeOutMasses.set(edge.src, _nodeOutMasses.get(edge.src) || 0 + pr);

Umm, I believe that addition is actually higher precedence than ||. so i think this code is wrong. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_Precedence#:~:text=Operator%20precedence%20determines%20how%20operators,of%20operators%20with%20lower%20precedence.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import type {TimestampMs} from "../util/timestamp";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +mint: number,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++const CORE_NODE_PREFIX = NodeAddress.fromParts(["sourcecred", "core"]);++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED");+const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+const EPOCH_RADIATION = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_RADIATION");++// Prefixes for seed edges.+const CONTRIBUTION_RADIATION = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "CONTRIBUTION_RADIATION",+]);+const SEED_MINT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_MINT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(a, prefix)` for some+  // element `prefix` of `what`.+  +what: $ReadOnlyArray<NodeAddressT>,+  // Transition probability for payout edges from epoch nodes to their+  // owners.+  +beta: TransitionProbability,+  // Transition probability for webbing edges from an epoch node to the+  // next epoch node for the same owner.+  +gammaForward: TransitionProbability,+  +gammaBackward: TransitionProbability,+|};+export type SeedOptions = {|+  +alpha: TransitionProbability,+|};++const COMPAT_INFO = {type: "sourcecred/markovProcessGraph", version: "0.1.0"};++export type MarkovProcessGraphJSON = Compatible<{|+  +nodes: {|+[NodeAddressT]: MarkovNode|},+  +edges: {|+[MarkovEdgeAddressT]: MarkovEdge|},+  +scoringAddresses: $ReadOnlyArray<NodeAddressT>,+|}>;++export class MarkovProcessGraph {+  _nodes: Map<NodeAddressT, MarkovNode>;+  _edges: Map<MarkovEdgeAddressT, MarkovEdge>;+  _scoringAddresses: Set<NodeAddressT>;++  constructor(+    nodes: Map<NodeAddressT, MarkovNode>,+    edges: Map<MarkovEdgeAddressT, MarkovEdge>,+    scoringAddresses: Set<NodeAddressT>+  ) {+    this._nodes = nodes;+    this._edges = edges;+    this._scoringAddresses = scoringAddresses;+  }++  static new(+    wg: WeightedGraphT,+    fibration: FibrationOptions,+    seed: SeedOptions+  ) {+    const _nodes = new Map();+    const _edges = new Map();+    const _scoringAddresses = _findScoringAddresses(wg.graph, fibration.what);++    // _nodeOutMasses[a] = sum(e.pr for e in edges if e.src == a)+    // Used for computing remainder-to-seed edges.+    const _nodeOutMasses = new Map();++    const epochTransitionRemainder = (() => {+      const {alpha} = seed;+      const {beta, gammaForward, gammaBackward} = fibration;+      if (beta < 0 || gammaForward < 0 || gammaBackward < 0) {+        throw new Error(+          "Negative transition probability: " ++            [beta, gammaForward, gammaBackward].join(" or ")+        );+      }+      const result = 1 - (alpha + beta + gammaForward + gammaBackward);+      if (result < 0) {+        throw new Error("Overlarge transition probability: " + (1 - result));+      }+      return result;+    })();++    const timeBoundaries = (() => {+      const edgeTimestamps = Array.from(+        wg.graph.edges({showDangling: false})+      ).map((x) => x.timestampMs);+      const start = min(edgeTimestamps);+      const end = max(edgeTimestamps);+      const boundaries = weekIntervals(start, end).map((x) => x.startTimeMs);+      return [-Infinity, ...boundaries, Infinity];+    })();++    const addNode = (node: MarkovNode) => {+      if (_nodes.has(node.address)) {+        throw new Error("Node conflict: " + node.address);+      }+      _nodes.set(node.address, node);+    };+    const addEdge = (edge: MarkovEdge) => {+      const mae = rawEdgeAddress(edge);+      if (_edges.has(mae)) {+        throw new Error("Edge conflict: " + mae);+      }+      const pr = edge.transitionProbability;+      if (pr < 0 || pr > 1) {+        const name = MarkovEdgeAddress.toString(mae);+        throw new Error(`Invalid transition probability for ${name}: ${pr}`);+      }+      _edges.set(mae, edge);+      _nodeOutMasses.set(edge.src, _nodeOutMasses.get(edge.src) || 0 + pr);+    };++    // Add seed node+    addNode({+      address: SEED_ADDRESS,+      description: SEED_DESCRIPTION,+      mint: 0,+    });++    // Add graph nodes+    const nwe = nodeWeightEvaluator(wg.weights);+    for (const node of wg.graph.nodes()) {+      const weight = nwe(node.address);+      if (weight < 0) {+        const name = NodeAddress.toString(node.address);+        throw new Error(`Negative node weight for ${name}: ${weight}`);+      }+      addNode({+        address: node.address,+        description: node.description,+        mint: weight,+      });+    }++    // Add epoch nodes, payout edges, and epoch webbing+    for (const scoringAddress of _scoringAddresses) {+      let lastBoundary = null;+      for (const boundary of timeBoundaries) {+        const thisEpoch = epochNodeAddressToRaw({+          type: "EPOCH_NODE",+          owner: scoringAddress,+          epochStart: boundary,+        });+        addNode({+          address: thisEpoch,+          description: `Epoch starting ${boundary} ms past epoch`,+          mint: 0,+        });+        addEdge({+          address: EdgeAddress.append(+            EPOCH_PAYOUT,+            String(boundary),+            ...NodeAddress.toParts(scoringAddress)+          ),+          reversed: false,+          src: thisEpoch,+          dst: scoringAddress,+          transitionProbability: fibration.beta,+        });+        if (lastBoundary != null) {+          const lastEpoch = epochNodeAddressToRaw({+            type: "EPOCH_NODE",+            owner: scoringAddress,+            epochStart: lastBoundary,+          });+          const webAddress = EdgeAddress.append(+            EPOCH_WEBBING,+            String(boundary),+            ...NodeAddress.toParts(scoringAddress)+          );+          addEdge({+            address: webAddress,+            reversed: false,+            src: lastEpoch,+            dst: thisEpoch,+            transitionProbability: fibration.gammaForward,+          });+          addEdge({+            address: webAddress,+            reversed: true,+            src: thisEpoch,+            dst: lastEpoch,+            transitionProbability: fibration.gammaBackward,+          });+        }+        lastBoundary = boundary;+      }+    }++    // Add minting edges, from the seed to positive-weight graph nodes+    {+      let totalNodeWeight = 0.0;+      const positiveNodeWeights: Map<NodeAddressT, number> = new Map();+      for (const {address, mint} of _nodes.values()) {+        if (mint > 0) {+          totalNodeWeight += mint;+          positiveNodeWeights.set(address, mint);+        }+      }+      if (!(totalNodeWeight > 0)) {+        throw new Error("No outflow from seed; add cred-minting nodes");+      }+      for (const [address, weight] of positiveNodeWeights) {+        addEdge({+          address: EdgeAddress.append(+            SEED_MINT,+            ...NodeAddress.toParts(address)+          ),+          reversed: false,+          src: SEED_ADDRESS,+          dst: address,+          transitionProbability: weight / totalNodeWeight,+        });+      }+    }++    /**+     * Find an epoch node, or just the original node if it's not a+     * scoring address.+     */+    const rewriteEpochNode = (+      address: NodeAddressT,+      edgeTimestampMs: TimestampMs+    ): NodeAddressT => {+      if (!_scoringAddresses.has(address)) {+        return address;+      }+      const epochEndIndex = sortedIndex(timeBoundaries, edgeTimestampMs);+      const epochStartIndex = epochEndIndex - 1;+      const epochTimestampMs = timeBoundaries[epochStartIndex];+      return epochNodeAddressToRaw({+        type: "EPOCH_NODE",+        owner: address,+        epochStart: epochTimestampMs,+      });+    };

Would be nice if we can get unit testing of this function in particular. Feels like if it were off-by-one our current test plan wouldn't catch it.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import type {TimestampMs} from "../util/timestamp";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +mint: number,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++const CORE_NODE_PREFIX = NodeAddress.fromParts(["sourcecred", "core"]);++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED");+const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+const EPOCH_RADIATION = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_RADIATION");++// Prefixes for seed edges.+const CONTRIBUTION_RADIATION = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "CONTRIBUTION_RADIATION",+]);+const SEED_MINT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_MINT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(a, prefix)` for some+  // element `prefix` of `what`.+  +what: $ReadOnlyArray<NodeAddressT>,+  // Transition probability for payout edges from epoch nodes to their+  // owners.+  +beta: TransitionProbability,+  // Transition probability for webbing edges from an epoch node to the+  // next epoch node for the same owner.+  +gammaForward: TransitionProbability,+  +gammaBackward: TransitionProbability,+|};+export type SeedOptions = {|+  +alpha: TransitionProbability,+|};++const COMPAT_INFO = {type: "sourcecred/markovProcessGraph", version: "0.1.0"};++export type MarkovProcessGraphJSON = Compatible<{|+  +nodes: {|+[NodeAddressT]: MarkovNode|},+  +edges: {|+[MarkovEdgeAddressT]: MarkovEdge|},+  +scoringAddresses: $ReadOnlyArray<NodeAddressT>,+|}>;++export class MarkovProcessGraph {+  _nodes: Map<NodeAddressT, MarkovNode>;+  _edges: Map<MarkovEdgeAddressT, MarkovEdge>;+  _scoringAddresses: Set<NodeAddressT>;++  constructor(+    nodes: Map<NodeAddressT, MarkovNode>,+    edges: Map<MarkovEdgeAddressT, MarkovEdge>,+    scoringAddresses: Set<NodeAddressT>+  ) {+    this._nodes = nodes;+    this._edges = edges;+    this._scoringAddresses = scoringAddresses;+  }++  static new(+    wg: WeightedGraphT,+    fibration: FibrationOptions,+    seed: SeedOptions+  ) {+    const _nodes = new Map();+    const _edges = new Map();+    const _scoringAddresses = _findScoringAddresses(wg.graph, fibration.what);++    // _nodeOutMasses[a] = sum(e.pr for e in edges if e.src == a)+    // Used for computing remainder-to-seed edges.+    const _nodeOutMasses = new Map();++    const epochTransitionRemainder = (() => {+      const {alpha} = seed;+      const {beta, gammaForward, gammaBackward} = fibration;+      if (beta < 0 || gammaForward < 0 || gammaBackward < 0) {+        throw new Error(+          "Negative transition probability: " ++            [beta, gammaForward, gammaBackward].join(" or ")+        );+      }+      const result = 1 - (alpha + beta + gammaForward + gammaBackward);+      if (result < 0) {+        throw new Error("Overlarge transition probability: " + (1 - result));+      }+      return result;+    })();++    const timeBoundaries = (() => {+      const edgeTimestamps = Array.from(+        wg.graph.edges({showDangling: false})+      ).map((x) => x.timestampMs);+      const start = min(edgeTimestamps);+      const end = max(edgeTimestamps);+      const boundaries = weekIntervals(start, end).map((x) => x.startTimeMs);+      return [-Infinity, ...boundaries, Infinity];+    })();++    const addNode = (node: MarkovNode) => {+      if (_nodes.has(node.address)) {+        throw new Error("Node conflict: " + node.address);+      }+      _nodes.set(node.address, node);+    };+    const addEdge = (edge: MarkovEdge) => {+      const mae = rawEdgeAddress(edge);+      if (_edges.has(mae)) {+        throw new Error("Edge conflict: " + mae);+      }+      const pr = edge.transitionProbability;+      if (pr < 0 || pr > 1) {+        const name = MarkovEdgeAddress.toString(mae);+        throw new Error(`Invalid transition probability for ${name}: ${pr}`);+      }+      _edges.set(mae, edge);+      _nodeOutMasses.set(edge.src, _nodeOutMasses.get(edge.src) || 0 + pr);+    };++    // Add seed node+    addNode({+      address: SEED_ADDRESS,+      description: SEED_DESCRIPTION,+      mint: 0,+    });++    // Add graph nodes+    const nwe = nodeWeightEvaluator(wg.weights);+    for (const node of wg.graph.nodes()) {+      const weight = nwe(node.address);+      if (weight < 0) {+        const name = NodeAddress.toString(node.address);+        throw new Error(`Negative node weight for ${name}: ${weight}`);+      }+      addNode({+        address: node.address,+        description: node.description,+        mint: weight,+      });+    }++    // Add epoch nodes, payout edges, and epoch webbing+    for (const scoringAddress of _scoringAddresses) {+      let lastBoundary = null;+      for (const boundary of timeBoundaries) {+        const thisEpoch = epochNodeAddressToRaw({+          type: "EPOCH_NODE",+          owner: scoringAddress,+          epochStart: boundary,+        });+        addNode({+          address: thisEpoch,+          description: `Epoch starting ${boundary} ms past epoch`,+          mint: 0,+        });+        addEdge({+          address: EdgeAddress.append(+            EPOCH_PAYOUT,+            String(boundary),+            ...NodeAddress.toParts(scoringAddress)+          ),+          reversed: false,+          src: thisEpoch,+          dst: scoringAddress,+          transitionProbability: fibration.beta,+        });+        if (lastBoundary != null) {+          const lastEpoch = epochNodeAddressToRaw({+            type: "EPOCH_NODE",+            owner: scoringAddress,+            epochStart: lastBoundary,+          });+          const webAddress = EdgeAddress.append(+            EPOCH_WEBBING,+            String(boundary),+            ...NodeAddress.toParts(scoringAddress)+          );+          addEdge({+            address: webAddress,+            reversed: false,+            src: lastEpoch,+            dst: thisEpoch,+            transitionProbability: fibration.gammaForward,+          });+          addEdge({+            address: webAddress,+            reversed: true,+            src: thisEpoch,+            dst: lastEpoch,+            transitionProbability: fibration.gammaBackward,+          });+        }+        lastBoundary = boundary;+      }+    }++    // Add minting edges, from the seed to positive-weight graph nodes+    {+      let totalNodeWeight = 0.0;+      const positiveNodeWeights: Map<NodeAddressT, number> = new Map();+      for (const {address, mint} of _nodes.values()) {+        if (mint > 0) {+          totalNodeWeight += mint;+          positiveNodeWeights.set(address, mint);+        }+      }+      if (!(totalNodeWeight > 0)) {+        throw new Error("No outflow from seed; add cred-minting nodes");+      }+      for (const [address, weight] of positiveNodeWeights) {+        addEdge({+          address: EdgeAddress.append(+            SEED_MINT,+            ...NodeAddress.toParts(address)+          ),+          reversed: false,+          src: SEED_ADDRESS,+          dst: address,+          transitionProbability: weight / totalNodeWeight,+        });+      }+    }++    /**+     * Find an epoch node, or just the original node if it's not a+     * scoring address.+     */+    const rewriteEpochNode = (+      address: NodeAddressT,+      edgeTimestampMs: TimestampMs+    ): NodeAddressT => {+      if (!_scoringAddresses.has(address)) {+        return address;+      }+      const epochEndIndex = sortedIndex(timeBoundaries, edgeTimestampMs);+      const epochStartIndex = epochEndIndex - 1;+      const epochTimestampMs = timeBoundaries[epochStartIndex];+      return epochNodeAddressToRaw({+        type: "EPOCH_NODE",+        owner: address,+        epochStart: epochTimestampMs,+      });+    };++    // Add graph edges. First, split by direction.+    type _UnidirectionalGraphEdge = {|+      +address: EdgeAddressT,+      +reversed: boolean,+      +src: NodeAddressT,+      +dst: NodeAddressT,+      +timestamp: TimestampMs,+      +weight: number,+    |};+    const unidirectionalGraphEdges = function* (): Iterator<_UnidirectionalGraphEdge> {+      const ewe = edgeWeightEvaluator(wg.weights);+      for (const edge of (function* () {+        for (const edge of wg.graph.edges({showDangling: false})) {+          const weight = ewe(edge.address);+          yield {+            address: edge.address,+            reversed: false,+            src: edge.src,+            dst: edge.dst,+            timestamp: edge.timestampMs,+            weight: weight.forwards,+          };+          yield {+            address: edge.address,+            reversed: true,+            src: edge.dst,+            dst: edge.src,+            timestamp: edge.timestampMs,+            weight: weight.backwards,+          };+        }+      })()) {+        if (edge.weight > 0) {+          yield edge;+        }+      }+    };++    const srcNodes: Map<+      NodeAddressT /* domain: (nodes with out-edges) U (epoch nodes) */,+      {totalOutWeight: number, outEdges: _UnidirectionalGraphEdge[]}+    > = new Map();+    for (const graphEdge of unidirectionalGraphEdges()) {+      const src = rewriteEpochNode(graphEdge.src, graphEdge.timestamp);+      let datum = srcNodes.get(src);+      if (datum == null) {+        datum = {totalOutWeight: 0, outEdges: []};+        srcNodes.set(src, datum);+      }+      datum.totalOutWeight += graphEdge.weight;+      datum.outEdges.push(graphEdge);+    }+    for (const [src, {totalOutWeight, outEdges}] of srcNodes) {+      const totalOutPr = NodeAddress.hasPrefix(src, EPOCH_PREFIX)+        ? epochTransitionRemainder+        : 1 - seed.alpha;+      for (const outEdge of outEdges) {+        const pr = (outEdge.weight / totalOutWeight) * totalOutPr;+        addEdge({+          address: outEdge.address,+          reversed: outEdge.reversed,+          src: rewriteEpochNode(outEdge.src, outEdge.timestamp),+          dst: rewriteEpochNode(outEdge.dst, outEdge.timestamp),+          transitionProbability: pr,+        });+      }+    }++    // Add radiation edges+    for (const node of _nodes.values()) {+      if (node.address === SEED_ADDRESS) continue;+      let type;+      if (NodeAddress.hasPrefix(node.address, EPOCH_PREFIX)) {+        type = EPOCH_RADIATION;+      } else if (NodeAddress.hasPrefix(node.address, CORE_NODE_PREFIX)) {+        throw new Error(+          "invariant violation: unknown core node: " ++            NodeAddress.toString(node.address)+        );+      } else {+        type = CONTRIBUTION_RADIATION;+      }+      addEdge({+        address: EdgeAddress.append(type, ...NodeAddress.toParts(node.address)),+        reversed: false,+        src: node.address,+        dst: SEED_ADDRESS,+        transitionProbability:+          1 - NullUtil.orElse(_nodeOutMasses.get(node.address), 0),+      });+    }++    return new MarkovProcessGraph(_nodes, _edges, _scoringAddresses);+  }++  scoringAddresses(): Set<NodeAddressT> {+    return new Set(this._scoringAddresses);+  }++  node(address: NodeAddressT): MarkovNode | null {+    NodeAddress.assertValid(address);+    return this._nodes.get(address) || null;+  }++  *nodes(options?: {|+prefix: NodeAddressT|}): Iterator<MarkovNode> {+    const prefix = options ? options.prefix : NodeAddress.empty;+    for (const node of this._nodes.values()) {+      if (NodeAddress.hasPrefix(node.address, prefix)) {+        yield node;+      }+    }+  }++  *edges(): Iterator<MarkovEdge> {+    for (const edge of this._edges.values()) {+      yield edge;+    }+  }++  *inNeighbors(nodeAddress: NodeAddressT): Iterator<MarkovEdge> {+    for (const edge of this._edges.values()) {+      if (edge.dst !== nodeAddress) {+        continue;+      }+      yield edge;+    }+  }++  toMarkovChain(): OrderedSparseMarkovChain {+    const nodeOrder = Array.from(this._nodes.keys()).sort();+    const nodeIndex: Map<+      NodeAddressT,+      number /* index into nodeOrder */+    > = new Map();+    nodeOrder.forEach((n, i) => {+      nodeIndex.set(n, i);+    });++    const inNeighbors: Map<NodeAddressT, MarkovEdge[]> = new Map();+    for (const edge of this._edges.values()) {+      MapUtil.pushValue(inNeighbors, edge.dst, edge);+    }++    const chain = nodeOrder.map((addr) => {+      const inEdges = NullUtil.orElse(inNeighbors.get(addr), []);+      const inDegree = inEdges.length;+      const neighbor = new Uint32Array(inDegree);+      const weight = new Float64Array(inDegree);+      inEdges.forEach((e, i) => {+        // Note: We don't group-by src, so there may be multiple `j`

It would be an efficiency improvement to collapse these together, right? Smaller/denser markov chain that is semantically equivalent. I dont think this happens much in practice so no change requested.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import type {TimestampMs} from "../util/timestamp";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +mint: number,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++const CORE_NODE_PREFIX = NodeAddress.fromParts(["sourcecred", "core"]);++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED");+const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+const EPOCH_RADIATION = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_RADIATION");++// Prefixes for seed edges.+const CONTRIBUTION_RADIATION = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "CONTRIBUTION_RADIATION",+]);+const SEED_MINT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_MINT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(a, prefix)` for some+  // element `prefix` of `what`.+  +what: $ReadOnlyArray<NodeAddressT>,

n.b. later I think we'll want to make this take a set of scoring node addresses, rather than prefixes. that way we can filter out bot accounts (which cannot be filtered based on address, since it's identity metadata not in the node address.) fine for now.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import type {TimestampMs} from "../util/timestamp";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +mint: number,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++const CORE_NODE_PREFIX = NodeAddress.fromParts(["sourcecred", "core"]);++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED");+const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+const EPOCH_RADIATION = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_RADIATION");++// Prefixes for seed edges.+const CONTRIBUTION_RADIATION = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "CONTRIBUTION_RADIATION",+]);+const SEED_MINT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_MINT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(a, prefix)` for some+  // element `prefix` of `what`.+  +what: $ReadOnlyArray<NodeAddressT>,+  // Transition probability for payout edges from epoch nodes to their+  // owners.+  +beta: TransitionProbability,+  // Transition probability for webbing edges from an epoch node to the+  // next epoch node for the same owner.+  +gammaForward: TransitionProbability,+  +gammaBackward: TransitionProbability,+|};+export type SeedOptions = {|+  +alpha: TransitionProbability,+|};++const COMPAT_INFO = {type: "sourcecred/markovProcessGraph", version: "0.1.0"};++export type MarkovProcessGraphJSON = Compatible<{|+  +nodes: {|+[NodeAddressT]: MarkovNode|},+  +edges: {|+[MarkovEdgeAddressT]: MarkovEdge|},+  +scoringAddresses: $ReadOnlyArray<NodeAddressT>,+|}>;++export class MarkovProcessGraph {+  _nodes: Map<NodeAddressT, MarkovNode>;+  _edges: Map<MarkovEdgeAddressT, MarkovEdge>;+  _scoringAddresses: Set<NodeAddressT>;++  constructor(+    nodes: Map<NodeAddressT, MarkovNode>,+    edges: Map<MarkovEdgeAddressT, MarkovEdge>,+    scoringAddresses: Set<NodeAddressT>+  ) {+    this._nodes = nodes;+    this._edges = edges;+    this._scoringAddresses = scoringAddresses;+  }++  static new(+    wg: WeightedGraphT,+    fibration: FibrationOptions,+    seed: SeedOptions+  ) {+    const _nodes = new Map();+    const _edges = new Map();+    const _scoringAddresses = _findScoringAddresses(wg.graph, fibration.what);++    // _nodeOutMasses[a] = sum(e.pr for e in edges if e.src == a)+    // Used for computing remainder-to-seed edges.+    const _nodeOutMasses = new Map();++    const epochTransitionRemainder = (() => {+      const {alpha} = seed;+      const {beta, gammaForward, gammaBackward} = fibration;+      if (beta < 0 || gammaForward < 0 || gammaBackward < 0) {+        throw new Error(+          "Negative transition probability: " ++            [beta, gammaForward, gammaBackward].join(" or ")+        );+      }+      const result = 1 - (alpha + beta + gammaForward + gammaBackward);+      if (result < 0) {+        throw new Error("Overlarge transition probability: " + (1 - result));+      }+      return result;+    })();++    const timeBoundaries = (() => {+      const edgeTimestamps = Array.from(+        wg.graph.edges({showDangling: false})+      ).map((x) => x.timestampMs);+      const start = min(edgeTimestamps);+      const end = max(edgeTimestamps);+      const boundaries = weekIntervals(start, end).map((x) => x.startTimeMs);+      return [-Infinity, ...boundaries, Infinity];+    })();++    const addNode = (node: MarkovNode) => {+      if (_nodes.has(node.address)) {+        throw new Error("Node conflict: " + node.address);+      }+      _nodes.set(node.address, node);+    };+    const addEdge = (edge: MarkovEdge) => {+      const mae = rawEdgeAddress(edge);+      if (_edges.has(mae)) {+        throw new Error("Edge conflict: " + mae);+      }+      const pr = edge.transitionProbability;+      if (pr < 0 || pr > 1) {+        const name = MarkovEdgeAddress.toString(mae);+        throw new Error(`Invalid transition probability for ${name}: ${pr}`);+      }+      _edges.set(mae, edge);+      _nodeOutMasses.set(edge.src, _nodeOutMasses.get(edge.src) || 0 + pr);+    };++    // Add seed node+    addNode({+      address: SEED_ADDRESS,+      description: SEED_DESCRIPTION,+      mint: 0,+    });++    // Add graph nodes+    const nwe = nodeWeightEvaluator(wg.weights);+    for (const node of wg.graph.nodes()) {+      const weight = nwe(node.address);+      if (weight < 0) {+        const name = NodeAddress.toString(node.address);+        throw new Error(`Negative node weight for ${name}: ${weight}`);+      }+      addNode({

maybe check that node is a scoring node implies that the weight is zero?

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import type {TimestampMs} from "../util/timestamp";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +mint: number,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++const CORE_NODE_PREFIX = NodeAddress.fromParts(["sourcecred", "core"]);++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.append(CORE_NODE_PREFIX, "SEED");+const SEED_DESCRIPTION = "\u{1f331}"; // U+1F331 SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.append(CORE_NODE_PREFIX, "EPOCH");++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+const EPOCH_RADIATION = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_RADIATION");

nit: we could make our addresses a little bit more concise by making the prefix be "sourcecred/core/epoch" and then the types be PAYOUT, WEBBING, RADIATION.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import type {TimestampMs} from "../util/timestamp";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +mint: number,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.

nit: i think "sum to 1" is more accessible than "sum to unity"

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;

Again, would prefer "underlying edges" or such.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);

It's confusing that we refer to users as a subtype of "contribution nodes" (from a nomenclature perspective). Maybe we can call these "underlying nodes", or "input graph nodes" or such? Since this category includes every node that was in the input weighted graph.

wchargin

comment created time in 8 days

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *radiation edges* edges from nodes to the seed node;+ *   - *minting* edges from the seed node to cred-minting nodes;

inconsistent italicization (include edges inside the italics)

wchargin

comment created time in 8 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *seed-in* / "redistribution" / "radiation" edges from nodes to+ *     the seed node;+ *   - *seed-out* / "minting" edges from the seed node to cred-minting+ *     nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {type NodeWeight} from "./weights";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TimestampMs = number;+export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +weight: NodeWeight,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.fromParts(["sourcecred", "core", "SEED"]);+const SEED_DESCRIPTION = "\u{1f331}"; // SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.fromParts(["sourcecred", "core", "EPOCH"]);++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+// Prefixes for seed edges.+const SEED_IN = EdgeAddress.fromParts(["sourcecred", "core", "SEED_IN"]);+const SEED_OUT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_OUT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(node, prefix)` for+  // some element `prefix` of `what`.+  +what: $ReadOnlyArray<NodeAddressT>,+  // Transition probability for payout edges from epoch nodes to their+  // owners.+  +beta: TransitionProbability,+  // Transition probability for webbing edges from an epoch node to the+  // next epoch node for the same owner.+  +gammaForward: TransitionProbability,+  +gammaBackward: TransitionProbability,+|};+export type SeedOptions = {|+  +alpha: TransitionProbability,+|};++const COMPAT_INFO = {type: "sourcecred/markovProcessGraph", version: "0.1.0"};++export type MarkovProcessGraphJSON = Compatible<{|+  +nodes: {|+[NodeAddressT]: MarkovNode|},+  +edges: {|+[MarkovEdgeAddressT]: MarkovEdge|},+  +scoringAddresses: $ReadOnlyArray<NodeAddressT>,+|}>;++export class MarkovProcessGraph {+  _nodes: Map<NodeAddressT, MarkovNode>;+  _edges: Map<MarkovEdgeAddressT, MarkovEdge>;+  _scoringAddresses: Set<NodeAddressT>;++  constructor(+    nodes: Map<NodeAddressT, MarkovNode>,+    edges: Map<MarkovEdgeAddressT, MarkovEdge>,+    scoringAddresses: Set<NodeAddressT>+  ) {+    this._nodes = nodes;+    this._edges = edges;+    this._scoringAddresses = scoringAddresses;

I really like that we take an explicit set of scoring addresses, rather than prefix matchers.

wchargin

comment created time in 8 days

PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

credrank: add Markov process graph and basic CLI

+// @flow++/**+ * Data structure representing a particular kind of Markov process, as+ * kind of a middle ground between the semantic SourceCred graph (in the+ * `core/graph` module) and a literal transition matrix. Unlike the core+ * graph, edges in a Markov process graph are unidirectional, edge+ * weights are raw transition probabilities (which must sum to 1) rather+ * than unnormalized weights, and there are no dangling edges. Unlike a+ * fully general transition matrix, parallel edges are still reified,+ * not collapsed; nodes have weights, representing sources of flow, and+ * a few SourceCred-specific concepts are made first-class:+ * specifically, cred minting and time period fibration. The+ * "teleportation vector" from PageRank is also made explicit via the+ * "adjoined seed node" graph transformation strategy, so this data+ * structure can form well-defined Markov processes even from graphs+ * with nodes with no out-weight. Because the graph reifies the+ * teleportation and temporal fibration, the associated parameters are+ * "baked in" to weights of the Markov process graph.+ *+ * We use the term "fibration" to refer to a graph transformation where+ * each scoring node is split into one node per epoch, and incident+ * edges are rewritten to point to the appropriate epoch nodes. The term+ * is vaguely inspired from the notion of a fiber bundle, though the+ * analogy is not precise.+ *+ * The Markov process graphs in this module have three kinds of nodes:+ *+ *   - *contribution nodes*, which are in 1-to-1 correspondence with the+ *     nodes in the underlying core graph (including users);+ *   - *epoch nodes*, which are created for each time period for each+ *     scoring node; and+ *   - the *seed node*, which reifies the teleportation vector and+ *     forces well-definedness and ergodicity of the Markov process (for+ *     nonzero alpha, and assuming that there is at least one edge in+ *     the underlying graph).+ *+ * The edges include:+ *+ *   - *contribution edges* between nodes in the underlying graph, which+ *     are lifted to their corresponding contribution nodes or to epoch+ *     nodes if either endpoint has been fibrated;+ *   - *seed-in* / "redistribution" / "radiation" edges from nodes to+ *     the seed node;+ *   - *seed-out* / "minting" edges from the seed node to cred-minting+ *     nodes;+ *   - *webbing edges* between temporally adjacent epoch nodes; and+ *   - *payout edges* from an epoch node to its owner (a scoring node).+ *+ * A Markov process graph can be converted to a pure Markov chain for+ * spectral analysis via the `toMarkovChain` method.+ */++import {max, min} from "d3-array";+import {weekIntervals} from "./interval";+import sortedIndex from "lodash.sortedindex";+import {makeAddressModule, type AddressModule} from "./address";+import {+  type NodeAddressT,+  NodeAddress,+  type EdgeAddressT,+  EdgeAddress,+  type Graph,+} from "./graph";+import {type WeightedGraph as WeightedGraphT} from "./weightedGraph";+import {type NodeWeight} from "./weights";+import {+  nodeWeightEvaluator,+  edgeWeightEvaluator,+} from "./algorithm/weightEvaluator";+import {toCompat, fromCompat, type Compatible} from "../util/compat";+import * as NullUtil from "../util/null";+import * as MapUtil from "../util/map";+import {type SparseMarkovChain} from "./algorithm/markovChain";++export type TimestampMs = number;+export type TransitionProbability = number;++export type MarkovNode = {|+  // Node address, unique within a Markov process graph. This is either+  // the address of a contribution node or an address under the+  // `sourcecred/core` namespace.+  +address: NodeAddressT,+  // Markdown source description, as in `Node` from `core/graph`.+  +description: string,+  // Amount of cred to mint at this node.+  +weight: NodeWeight,+|};+export type MarkovEdge = {|+  // Address of the underlying edge. Note that this attribute alone does+  // not uniquely identify an edge in the Markov process graph; the+  // primary key is `(address, reversed)`, not just `address`. For edges+  // not in the underlying graph (e.g., fibration edges), this will be+  // an address under the `sourcecred/core` namespace.+  +address: EdgeAddressT,+  // If this came from an underlying graph edge or an epoch webbing+  // edge, have its `src` and `dst` been swapped in the process of+  // handling the reverse component of a bidirectional edge?+  +reversed: boolean,+  // Source node at the Markov chain level.+  +src: NodeAddressT,+  // Destination node at the Markov chain level.+  +dst: NodeAddressT,+  // Transition probability: $Pr[X_{n+1} = dst | X_{n} = src]$. Must sum+  // to unity for a given `src`.+  +transitionProbability: TransitionProbability,+|};+export opaque type MarkovEdgeAddressT: string = string;+export const MarkovEdgeAddress: AddressModule<MarkovEdgeAddressT> = (makeAddressModule(+  {+    name: "MarkovEdgeAddress",+    nonce: "ME",+    otherNonces: new Map().set("N", "NodeAddress").set("E", "EdgeAddress"),+  }+): AddressModule<string>);++function rawEdgeAddress(edge: MarkovEdge): MarkovEdgeAddressT {+  return MarkovEdgeAddress.fromParts([+    edge.reversed ? "B" /* Backward */ : "F" /* Forward */,+    ...EdgeAddress.toParts(edge.address),+  ]);+}++export type OrderedSparseMarkovChain = {|+  +nodeOrder: $ReadOnlyArray<NodeAddressT>,+  +chain: SparseMarkovChain,+|};++// Address of a seed node. All graph nodes tithe $\alpha$ to this node,+// and this node flows out to nodes in proportion to their weight. This+// is also a node prefix for the "seed node" type, which contains only+// one node.+const SEED_ADDRESS = NodeAddress.fromParts(["sourcecred", "core", "SEED"]);+const SEED_DESCRIPTION = "\u{1f331}"; // SEEDLING++// Node address prefix for epoch nodes.+const EPOCH_PREFIX = NodeAddress.fromParts(["sourcecred", "core", "EPOCH"]);++export type EpochNodeAddress = {|+  +type: "EPOCH_NODE",+  +owner: NodeAddressT,+  +epochStart: TimestampMs,+|};++export function epochNodeAddressToRaw(addr: EpochNodeAddress): NodeAddressT {+  return NodeAddress.append(+    EPOCH_PREFIX,+    String(addr.epochStart),+    ...NodeAddress.toParts(addr.owner)+  );+}++export function epochNodeAddressFromRaw(addr: NodeAddressT): EpochNodeAddress {+  if (!NodeAddress.hasPrefix(addr, EPOCH_PREFIX)) {+    throw new Error("Not an epoch node address: " + NodeAddress.toString(addr));+  }+  const epochPrefixLength = NodeAddress.toParts(EPOCH_PREFIX).length;+  const parts = NodeAddress.toParts(addr);+  const epochStart = +parts[epochPrefixLength];+  const owner = NodeAddress.fromParts(parts.slice(epochPrefixLength + 1));+  return {+    type: "EPOCH_NODE",+    owner,+    epochStart,+  };+}++// Prefixes for fibration edges.+const FIBRATION_EDGE = EdgeAddress.fromParts([+  "sourcecred",+  "core",+  "fibration",+]);+const EPOCH_PAYOUT = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_PAYOUT");+const EPOCH_WEBBING = EdgeAddress.append(FIBRATION_EDGE, "EPOCH_WEBBING");+// Prefixes for seed edges.+const SEED_IN = EdgeAddress.fromParts(["sourcecred", "core", "SEED_IN"]);+const SEED_OUT = EdgeAddress.fromParts(["sourcecred", "core", "SEED_OUT"]);++export type FibrationOptions = {|+  // List of node prefixes for temporal fibration. A node with address+  // `a` will be fibrated if `NodeAddress.hasPrefix(node, prefix)` for

nit: s/node/a in the code example

wchargin

comment created time in 8 days

PullRequestReviewEvent

issue commentsourcecred/sourcecred

Measure "active participants"

It's more sibyll resistant …

Question about this: wouldn't having multiple identities and making each of them make a different contribution ultimately be self-defeating regardless of this threshold, since each identity's internal reputation stays lower than if you had just the one?

It depends on the specific incentives being gamed.

I think that using Cred is trivially more Sybil resistent than just counting "number of people who made at least one contribution", because obviously it's trivial to spam new accounts that each make one post or such, and game that metric.

So let's consider the much higher-effort attack of making multiple accounts, and having each account make real contributions. This attack would probably succeed, in that the attacker would successfully control multiple "active" accounts with nontrivial Cred. Would it be worth the effort? That depends on the attacker's goals and incentives.

If the attacker's goal is to maximize their Grain distributions, by maximizing their Cred, then the attack is probably not worthwhile. Having one account with k contributions will probably have slightly higher Cred overall than having k accounts with one contribution, and it's definitely less effort to make the one account. So people won't make Sybil accounts for the purpose of farming Grain.

However, suppose that we implement a mechanism where we make governance decisions with Cred-weighted quadratic voting. In that case, the voting power of k accounts with one contribution each might be materially higher than the voting power of one account with k contributions. So an attacker trying to get disproportionate voting power might be incentivized to create Sybils. And this is something we should think about if we start using quadratic voting, and we might want to incorporate mechanisms to mitigate this kind of attack.

In general, when contemplating any kind of security property (e.g. the property of Sybil resistance, or whether your system is secure against hackers), you need to think about it in terms of your threat model. It's easy to be robust against script kiddies trying to make an easy buck, much harder to be robust against motivated ideological opponents, or national governments.

decentralion

comment created time in 9 days

issue openedsourcecred/sourcecred

Create a system for retrieving identity ids from plugin-specific ids

We should add a first-class way to lookup identity ids given plugin-specific identifiers.

We can do this by extending the identity proposal type so it includes a plugin-specific id (just a string): https://github.com/sourcecred/sourcecred/blob/1b8c42a3519b8240e988f021e79b22b5918d367e/src/ledger/identityProposal.js#L38-L43

Then, when SourceCred core uses all the identity proposals to update the ledger, we could also populate a map from plugin-specific id to identity id, like follows:

export type PluginIdMap = Map<string, IdentityId>
export type PluginIds = Map<PluginName, PluginIdMap>

Prompted by @benoxmo, in discussion with @hammadj

created time in 9 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

push eventsourcecred/sourcecred

dependabot[bot]

commit sha 8d54bbc0f0ca5477303868a4e650fb99ddcea1f5

Bump better-sqlite3 from 7.1.0 to 7.1.1 (#2263) Bumps [better-sqlite3](https://github.com/JoshuaWise/better-sqlite3) from 7.1.0 to 7.1.1. - [Release notes](https://github.com/JoshuaWise/better-sqlite3/releases) - [Commits](https://github.com/JoshuaWise/better-sqlite3/compare/v7.1.0...v7.1.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

view details

push time in 11 days

delete branch sourcecred/sourcecred

delete branch : dependabot/npm_and_yarn/better-sqlite3-7.1.1

delete time in 11 days

PR merged sourcecred/sourcecred

Bump better-sqlite3 from 7.1.0 to 7.1.1 dependencies

Bumps better-sqlite3 from 7.1.0 to 7.1.1. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/JoshuaWise/better-sqlite3/commit/03f382a3ba1977feeb57ff1349c628b846d8b8b3"><code>03f382a</code></a> 7.1.1</li> <li><a href="https://github.com/JoshuaWise/better-sqlite3/commit/5cbcb1a5746de8edd521b3e2a020bd5788ac3b67"><code>5cbcb1a</code></a> upgraded to SQLite v3.33.0</li> <li>See full diff in <a href="https://github.com/JoshuaWise/better-sqlite3/compare/v7.1.0...v7.1.1">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

</details>

+3 -3

0 comment

1 changed file

dependabot[bot]

pr closed time in 11 days

PullRequestReviewEvent
PullRequestReviewEvent

push eventsourcecred/sourcecred

Dandelion Mané

commit sha 56e8e7d76cbf8b743b64dbd09dfab24972558bf6

Log SourceCred version on load (#2259) This commit prints the SourceCred version into the console on frontend load. This will be useful to help provide tech support for users, since we'll be able to find what version of SourceCred they'll be using. Test plan: Run `yarn start`, and check the console.

view details

push time in 11 days

delete branch sourcecred/sourcecred

delete branch : sourcecred-version-in-console

delete time in 11 days

PR merged sourcecred/sourcecred

Reviewers
Log SourceCred version on load

This commit prints the SourceCred version into the console on frontend load. This will be useful to help provide tech support for users, since we'll be able to find what version of SourceCred they'll be using.

Test plan: Run yarn start, and check the console.

+3 -0

0 comment

1 changed file

decentralion

pr closed time in 11 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

Mui explorer regressions and spacing

 export type Props = {| export class WeightSlider extends React.Component<Props> {   render() {     return (-      <label style={{display: "flex"}} title={this.props.description}>-        <span style={{flexGrow: 1}}>{this.props.name}</span>-        <input-          type="range"-          min={MIN_SLIDER}-          max={MAX_SLIDER}-          step={1}-          value={weightToSlider(this.props.weight)}-          onChange={(e) => {-            const weight: Weight = sliderToWeight(e.target.valueAsNumber);-            this.props.onChange(weight);-          }}-        />{" "}-        <span-          style={{minWidth: 45, display: "inline-block", textAlign: "right"}}-        >-          {formatWeight(this.props.weight)}-        </span>-      </label>+      <Tooltip title={this.props.description} placement="top">+        <Grid container justify="space-between">+          <Grid item xs={4}>+            {this.props.name}+          </Grid>+          <Grid item xs={5}>+            <Slider+              value={weightToSlider(this.props.weight)}+              min={MIN_SLIDER}+              max={MAX_SLIDER}+              step={1}+              valueLabelDisplay="off"+              onChange={(_, val) => {+                const weight: Weight = sliderToWeight(val);+                console.log({weight});

That's a reasonable suggestion. There are some cases where we want to do it deliberately, but we could just disable the lint rule for those lines.

befitsandpiper

comment created time in 11 days

PullRequestReviewEvent

push eventsourcecred/cred

Dandelion Mané

commit sha dca506cfac9154a34c294716073730f3d8e615cc

Fix: update lockfile

view details

push time in 11 days

issue commentsourcecred/sourcecred

Measure "active participants"

Thanks for the clarification, @KuraFire. In our dev meeting today, @hammadj and i decided that you should be considered active in an interval if your Cred in that interval was over some configurable threshold. It's more sibyll resistant and flexible.

decentralion

comment created time in 11 days

push eventsourcecred/cred

Dandelion Mané

commit sha 12586c9137f25c9602cf8912c731609005e5d92a

Use beta 16

view details

push time in 12 days

delete branch sourcecred/cred

delete branch : generated-ledger-1599959458

delete time in 12 days

push eventsourcecred/cred

github-actions[bot]

commit sha 69aa25461481800f14caa7522cb76b03c7f7ba7c

update calculated ledger (#106) Co-authored-by: credbot <credbot@users.noreply.github.com>

view details

push time in 12 days

PR merged sourcecred/cred

Scheduled grain distribution for week ending September 13th, 2020

This PR was auto-generated on 13-09-2020 to add the latest grain distribution to our instance.

yarn run v1.22.5 $ sourcecred grain Distributed 25,000g to 40 identities in 1 distributions Done in 1.57s.

+1 -0

0 comment

1 changed file

github-actions[bot]

pr closed time in 12 days

PullRequestReviewEvent

Pull request review commentsourcecred/sourcecred

Mui explorer regressions and spacing

 export type Props = {| export class WeightSlider extends React.Component<Props> {   render() {     return (-      <label style={{display: "flex"}} title={this.props.description}>-        <span style={{flexGrow: 1}}>{this.props.name}</span>-        <input-          type="range"-          min={MIN_SLIDER}-          max={MAX_SLIDER}-          step={1}-          value={weightToSlider(this.props.weight)}-          onChange={(e) => {-            const weight: Weight = sliderToWeight(e.target.valueAsNumber);-            this.props.onChange(weight);-          }}-        />{" "}-        <span-          style={{minWidth: 45, display: "inline-block", textAlign: "right"}}-        >-          {formatWeight(this.props.weight)}-        </span>-      </label>+      <Tooltip title={this.props.description} placement="top">+        <Grid container justify="space-between">+          <Grid item xs={4}>+            {this.props.name}+          </Grid>+          <Grid item xs={5}>+            <Slider+              value={weightToSlider(this.props.weight)}+              min={MIN_SLIDER}+              max={MAX_SLIDER}+              step={1}+              valueLabelDisplay="off"+              onChange={(_, val) => {+                const weight: Weight = sliderToWeight(val);+                console.log({weight});

looks like debug code

befitsandpiper

comment created time in 12 days

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

PR opened sourcecred/sourcecred

Reviewers
Log SourceCred version on load

This commit prints the SourceCred version into the console on frontend load. This will be useful to help provide tech support for users, since we'll be able to find what version of SourceCred they'll be using.

Test plan: Run yarn start, and check the console.

+3 -0

0 comment

1 changed file

pr created time in 12 days

create barnchsourcecred/sourcecred

branch : sourcecred-version-in-console

created branch time in 12 days

push eventsourcecred/sourcecred

dependabot[bot]

commit sha f175a6d9f89a04c316713b1b565921aab068d7e2

Bump eslint from 7.8.1 to 7.9.0 (#2254) Bumps [eslint](https://github.com/eslint/eslint) from 7.8.1 to 7.9.0. - [Release notes](https://github.com/eslint/eslint/releases) - [Changelog](https://github.com/eslint/eslint/blob/master/CHANGELOG.md) - [Commits](https://github.com/eslint/eslint/compare/v7.8.1...v7.9.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

view details

push time in 12 days

delete branch sourcecred/sourcecred

delete branch : dependabot/npm_and_yarn/eslint-7.9.0

delete time in 12 days

PR merged sourcecred/sourcecred

Bump eslint from 7.8.1 to 7.9.0 dependencies

Bumps eslint from 7.8.1 to 7.9.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/eslint/eslint/releases">eslint's releases</a>.</em></p> <blockquote> <h2>v7.9.0</h2> <ul> <li><a href="https://github.com/eslint/eslint/commit/3ca27004ece5016ba7aed775f01ad13bc9282296"><code>3ca2700</code></a> Fix: Corrected notice for invalid (:) plugin names (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13473">#13473</a>) (Josh Goldberg)</li> <li><a href="https://github.com/eslint/eslint/commit/fc5783d2ff9e3b0d7a1f9664928d49270b4a6c01"><code>fc5783d</code></a> Docs: Fix leaky anchors in v4 migration page (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13635">#13635</a>) (Timo Tijhof)</li> <li><a href="https://github.com/eslint/eslint/commit/f1d07f112be96c64dfdaa154aa9ac81985b16238"><code>f1d07f1</code></a> Docs: Provide install commands for Yarn (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13661">#13661</a>) (Nikita Baksalyar)</li> <li><a href="https://github.com/eslint/eslint/commit/29d1cdceedd6c056a39149723cf9ff2fbb260cbf"><code>29d1cdc</code></a> Fix: prefer-destructuring removes comments (refs <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13678">#13678</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13682">#13682</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/b4da0a7ca7995435bdfc116fd374eb0649470131"><code>b4da0a7</code></a> Docs: fix typo in working with plugins docs (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13683">#13683</a>) (啸生)</li> <li><a href="https://github.com/eslint/eslint/commit/6f87db7c318225e48ccbbf0bec8b3758ea839b82"><code>6f87db7</code></a> Update: fix id-length false negatives on Object.prototype property names (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13670">#13670</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/361ac4d895c15086fb4351d4dca1405b2fdc4bd5"><code>361ac4d</code></a> Fix: NonOctalDecimalIntegerLiteral is decimal integer (fixes <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13588">#13588</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13664">#13664</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/f260716695064e4b4193337107b60401bd4b3f20"><code>f260716</code></a> Docs: update outdated link (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13677">#13677</a>) (klkhan)</li> <li><a href="https://github.com/eslint/eslint/commit/5138c913c256e4266ffb68278783af45bf70af84"><code>5138c91</code></a> Docs: add missing eslint directive comments in no-await-in-loop (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13673">#13673</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/17b58b528df62bf96813d50c087cafdf83306810"><code>17b58b5</code></a> Docs: clarify correct example in no-return-await (fixes <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13656">#13656</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13657">#13657</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/9171f0a99bb4d7c53f109b1c2b215004a7c27713"><code>9171f0a</code></a> Chore: fix typo (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13660">#13660</a>) (Nitin Kumar)</li> <li><a href="https://github.com/eslint/eslint/commit/6d9f8fbb7ed4361b475fb50d04e6d25744d5b1a2"><code>6d9f8fb</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/97b0dd9a1af1ae4ae3857adcfe6eeac7837101ed"><code>97b0dd9</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/deab125fc9220dab43baeb32c6cf78942ad25a83"><code>deab125</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/bf2e367bf4f6fde9930af9de8b8d8bc3d8b5782f"><code>bf2e367</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/89292084bf91ba5ae5bf966c6c56fa3da139ce57"><code>8929208</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/eslint/eslint/blob/master/CHANGELOG.md">eslint's changelog</a>.</em></p> <blockquote> <p>v7.9.0 - September 12, 2020</p> <ul> <li><a href="https://github.com/eslint/eslint/commit/3ca27004ece5016ba7aed775f01ad13bc9282296"><code>3ca2700</code></a> Fix: Corrected notice for invalid (:) plugin names (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13473">#13473</a>) (Josh Goldberg)</li> <li><a href="https://github.com/eslint/eslint/commit/fc5783d2ff9e3b0d7a1f9664928d49270b4a6c01"><code>fc5783d</code></a> Docs: Fix leaky anchors in v4 migration page (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13635">#13635</a>) (Timo Tijhof)</li> <li><a href="https://github.com/eslint/eslint/commit/f1d07f112be96c64dfdaa154aa9ac81985b16238"><code>f1d07f1</code></a> Docs: Provide install commands for Yarn (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13661">#13661</a>) (Nikita Baksalyar)</li> <li><a href="https://github.com/eslint/eslint/commit/29d1cdceedd6c056a39149723cf9ff2fbb260cbf"><code>29d1cdc</code></a> Fix: prefer-destructuring removes comments (refs <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13678">#13678</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13682">#13682</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/b4da0a7ca7995435bdfc116fd374eb0649470131"><code>b4da0a7</code></a> Docs: fix typo in working with plugins docs (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13683">#13683</a>) (啸生)</li> <li><a href="https://github.com/eslint/eslint/commit/6f87db7c318225e48ccbbf0bec8b3758ea839b82"><code>6f87db7</code></a> Update: fix id-length false negatives on Object.prototype property names (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13670">#13670</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/361ac4d895c15086fb4351d4dca1405b2fdc4bd5"><code>361ac4d</code></a> Fix: NonOctalDecimalIntegerLiteral is decimal integer (fixes <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13588">#13588</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13664">#13664</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/f260716695064e4b4193337107b60401bd4b3f20"><code>f260716</code></a> Docs: update outdated link (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13677">#13677</a>) (klkhan)</li> <li><a href="https://github.com/eslint/eslint/commit/5138c913c256e4266ffb68278783af45bf70af84"><code>5138c91</code></a> Docs: add missing eslint directive comments in no-await-in-loop (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13673">#13673</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/17b58b528df62bf96813d50c087cafdf83306810"><code>17b58b5</code></a> Docs: clarify correct example in no-return-await (fixes <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13656">#13656</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13657">#13657</a>) (Milos Djermanovic)</li> <li><a href="https://github.com/eslint/eslint/commit/9171f0a99bb4d7c53f109b1c2b215004a7c27713"><code>9171f0a</code></a> Chore: fix typo (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13660">#13660</a>) (Nitin Kumar)</li> <li><a href="https://github.com/eslint/eslint/commit/6d9f8fbb7ed4361b475fb50d04e6d25744d5b1a2"><code>6d9f8fb</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/97b0dd9a1af1ae4ae3857adcfe6eeac7837101ed"><code>97b0dd9</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/deab125fc9220dab43baeb32c6cf78942ad25a83"><code>deab125</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/bf2e367bf4f6fde9930af9de8b8d8bc3d8b5782f"><code>bf2e367</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> <li><a href="https://github.com/eslint/eslint/commit/89292084bf91ba5ae5bf966c6c56fa3da139ce57"><code>8929208</code></a> Sponsors: Sync README with website (ESLint Jenkins)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/eslint/eslint/commit/022257a71b7579cf88cf3b8b936a696e8d2a09ed"><code>022257a</code></a> 7.9.0</li> <li><a href="https://github.com/eslint/eslint/commit/1f0a4ac09560c2dc93551210f3907cb9f833b868"><code>1f0a4ac</code></a> Build: changelog update for 7.9.0</li> <li><a href="https://github.com/eslint/eslint/commit/3ca27004ece5016ba7aed775f01ad13bc9282296"><code>3ca2700</code></a> Fix: Corrected notice for invalid (:) plugin names (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13473">#13473</a>)</li> <li><a href="https://github.com/eslint/eslint/commit/fc5783d2ff9e3b0d7a1f9664928d49270b4a6c01"><code>fc5783d</code></a> Docs: Fix leaky anchors in v4 migration page (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13635">#13635</a>)</li> <li><a href="https://github.com/eslint/eslint/commit/f1d07f112be96c64dfdaa154aa9ac81985b16238"><code>f1d07f1</code></a> Docs: Provide install commands for Yarn (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13661">#13661</a>)</li> <li><a href="https://github.com/eslint/eslint/commit/29d1cdceedd6c056a39149723cf9ff2fbb260cbf"><code>29d1cdc</code></a> Fix: prefer-destructuring removes comments (refs <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13678">#13678</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13682">#13682</a>)</li> <li><a href="https://github.com/eslint/eslint/commit/b4da0a7ca7995435bdfc116fd374eb0649470131"><code>b4da0a7</code></a> Docs: fix typo in working with plugins docs (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13683">#13683</a>)</li> <li><a href="https://github.com/eslint/eslint/commit/6f87db7c318225e48ccbbf0bec8b3758ea839b82"><code>6f87db7</code></a> Update: fix id-length false negatives on Object.prototype property names (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13">#13</a>...</li> <li><a href="https://github.com/eslint/eslint/commit/361ac4d895c15086fb4351d4dca1405b2fdc4bd5"><code>361ac4d</code></a> Fix: NonOctalDecimalIntegerLiteral is decimal integer (fixes <a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13588">#13588</a>) (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13664">#13664</a>)</li> <li><a href="https://github.com/eslint/eslint/commit/f260716695064e4b4193337107b60401bd4b3f20"><code>f260716</code></a> Docs: update outdated link (<a href="https://github-redirect.dependabot.com/eslint/eslint/issues/13677">#13677</a>)</li> <li>Additional commits viewable in <a href="https://github.com/eslint/eslint/compare/v7.8.1...v7.9.0">compare view</a></li> </ul> </details> <br />

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


<details> <summary>Dependabot commands and options</summary> <br />

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

</details>

+3 -3

0 comment

1 changed file

dependabot[bot]

pr closed time in 12 days

PullRequestReviewEvent

issue commentsourcecred/sourcecred

Issues with SC release process

I think we should just have a script, release.sh, which updates all the relevant files, creates a git tag, publishes to npm. Then after running that, the tag already exists, and the maintainer can write the release notes on GitHub. It's a two step process, but has little room for error (e.g. since the Git tag is created in release.sh, we don't have to worry about desync between the GitHub release and the npm release, even if someone merges during the process).

Also: the script should nag you to QA the release, maybe linking to a QA process checklist.

decentralion

comment created time in 12 days

issue openedsourcecred/sourcecred

Issues with SC release process

Issues with the current release flow:

  • We tag and then update package.json, which means if you check out the tag, it has the wrong version
  • We don't update version.js, so sourcecred --version will lie.
  • We turned off 2fa on npm package deploy, which gives us worse security around the npm package. (We still have 2fa for getting credentials.)

created time in 12 days

PullRequestReviewEvent
more