profile
viewpoint
isaacs isaacs GitHub Oakland CA http://blog.izs.me Principal Engineer at GitHub npm inventor, founder npm, Inc. Former Node BDFL. All opinions are my own. Literally all of them. I own them all. He/him

indutny/caine 142

Friendly butler

isaacs/abbrev-js 139

Like ruby's Abbrev module

isaacs/async-cache 116

Cache your async lookups and don't fetch the same thing more than necessary.

dawsbot/config-chain 96

Handle configuration once and for all

isaacs/block-stream 48

A stream of fixed-size blocks

isaacs/.vim 44

My vim settings

dominictarr/stream-punks 40

discussion repo for streams

isaacs/back-to-markdown.css 19

Turns any markdown editor into a WYSIWYG editor

isaacs/ahyane 11

A blog engine that builds html from text files.

isaacs/blog.izs.me 8

Gatsby app that powers my blog

push eventnpm/arborist

isaacs

commit sha df2a1faeca55c187107c539d52891b653205bde6

Refactor shrinkwrap for correctness and elegance Since the Shrinkwrap class was developed prior to figuring out exactly how it would be used in the various Arborist methods, some things ended up growing in a somewhat inelegant direction. - Nodes were fetching their resolved and integrity values in multiple places. - The Shrinkwrap internal data tried to keep up both the new and old data up to date as the tree changed, but this was not always possible. - Dependencies of link targets were not properly reflected in the legacy shrinkwrap metadata. - The Node class and buildIdealTree methods both had to dive deep into the internals of the YarnLock class, which should be an implementation detail of the Shrinkwrap class. Now, the Shrinkwrap object keeps a reference to the root tree node found at its own path. Rather than try to keep the legacy metadata in sync at all times, we _only_ build up the legacy shrinkwrap data when calling `this.commit()`, by walking the `node.children` and `node.target.children` maps appropriately. Also, `this.yarnLock` is updated on commit(), and a `checkYarnLock()` method is added to return the spec that should be fetched, and update the provided options object with resolved and integrity metadata. All of the logic for setting node metadata from the Shrinkwrap is now done in `Shrinkwrap.add(node)`, making it much simpler and harder to get wrong. This means that: - `yarn.lock` files are respected more fully (including their resolved and integrity expectations) - We don't have cases where both `extraneous` and `dev`/`optional` are set in the lockfile (which is always an error -- extraneous nodes cannot be either dev or optional!) - Link target dependencies are accurately reflected in the legacy shrinkwrap data, rather than creating invalid `../` entries. Fix: #82 Fix: #84

view details

push time in 12 hours

push eventnpm/arborist

claudiahdz

commit sha bb4f1a82436a997b69be8d5879cf2334e81b5896

feat: arborist dedupe()

view details

Ruy Adorno

commit sha cfede2e78b14512aae9f18bfed64e30f1b541bd7

test: dedupe test cases

view details

Ruy Adorno

commit sha 9d01b6deea9c08ced54d15c65e376a40936cddb0

chore: dedupe cleanup - added fixtures for dedupe reify-cases - tweaks and cleanup in lib trimmer/build-ideal-tree Fix: #66 PR-URL: https://github.com/npm/arborist/pull/87 Credit: @ruyadorno, @claudiahdz Close: #87 Reviewed-by: @isaacs, @ruyadorno

view details

isaacs

commit sha a1430ce566002dd0a4c64528bdd70cd0023103da

Rework approach for finding link target deps This change updates a few things, resulting in a much more resilient and reasonable approach to finding the dependencies of link targets, which is relevant for workspaces and for interpreting pnpm trees, and reduces the "load my whole dev folder" antipattern. Prior to this change, in loadActual, if a link target was found in a node_modules folder, then the parent of that node_modules folder would be loaded, along with its children. While this is seems like a reasonable way to handle pnpm trees, it has the following problems: 1. If you are developing a bunch of projects right in a `node_modules` folder, then it would sometimes result in loading ALL of your JS projects in the actualTree! This is rarely what you want, and had some surprising behavior when I encountered it myself. (I develop quite a lot of projects in a node_modules folder for convenience.) 2. If you have a situation where a package is _not_ in a node_modules folder, but it does have some missing dependencies which are in fact resolved by deps already in the tree, it would show up as missing. With this change, the loadActual process will walk up the parent directories of any top nodes, searching for any missing edges. If a folder is found at `${dir}/node_modules/${name}`, then a "dummy" node is created for the parent, such that the parent/fsParent resolution algorithm will find a hit and resolve to the named dep. It will not load any _other_ dependencies in the dummy node, however, and will not walk up higher than the nearest common ancestor of the project root and the link target. Furthermore, this uncovered a bug where setting the parent of a node to a new parent with the same `root` was causing it to unlist itself and all children from the root.meta and root.inventory, but only _relist_ itself, and not its children. This bug was previously not hit in test because the node demonstrating it was within a node_modules folder, and so was fully loaded along with all of its siblings. Lastly, because dummy nodes are never "extraneous" (they're just there as placeholders, they're not "real" anyway), had to update a few shrinkwrap files in fixtures to not get weird results. An upcoming refactor of the Shrinkwrap class will make this incorrect metadata less of a hazard moving forward.

view details

isaacs

commit sha a79da2851c9bed64865c9ea6c2718f5a787d0887

Refactor shrinkwrap for correctness and elegance Since the Shrinkwrap class was developed prior to figuring out exactly how it would be used in the various Arborist methods, some things ended up growing in a somewhat inelegant direction. - Nodes were fetching their resolved and integrity values in multiple places. - The Shrinkwrap internal data tried to keep up both the new and old data up to date as the tree changed, but this was not always possible. - Dependencies of link targets were not properly reflected in the legacy shrinkwrap metadata. - The Node class and buildIdealTree methods both had to dive deep into the internals of the YarnLock class, which should be an implementation detail of the Shrinkwrap class. Now, the Shrinkwrap object keeps a reference to the root tree node found at its own path. Rather than try to keep the legacy metadata in sync at all times, we _only_ build up the legacy shrinkwrap data when calling `this.commit()`, by walking the `node.children` and `node.target.children` maps appropriately. Also, `this.yarnLock` is updated on commit(), and a `checkYarnLock()` method is added to return the spec that should be fetched, and update the provided options object with resolved and integrity metadata. All of the logic for setting node metadata from the Shrinkwrap is now done in `Shrinkwrap.add(node)`, making it much simpler and harder to get wrong. This means that: - `yarn.lock` files are respected more fully (including their resolved and integrity expectations) - We don't have cases where both `extraneous` and `dev`/`optional` are set in the lockfile (which is always an error -- extraneous nodes cannot be either dev or optional!) - Link target dependencies are accurately reflected in the legacy shrinkwrap data, rather than creating invalid `../` entries.

view details

push time in 12 hours

push eventnpm/arborist

claudiahdz

commit sha bb4f1a82436a997b69be8d5879cf2334e81b5896

feat: arborist dedupe()

view details

Ruy Adorno

commit sha cfede2e78b14512aae9f18bfed64e30f1b541bd7

test: dedupe test cases

view details

Ruy Adorno

commit sha 9d01b6deea9c08ced54d15c65e376a40936cddb0

chore: dedupe cleanup - added fixtures for dedupe reify-cases - tweaks and cleanup in lib trimmer/build-ideal-tree Fix: #66 PR-URL: https://github.com/npm/arborist/pull/87 Credit: @ruyadorno, @claudiahdz Close: #87 Reviewed-by: @isaacs, @ruyadorno

view details

isaacs

commit sha a1430ce566002dd0a4c64528bdd70cd0023103da

Rework approach for finding link target deps This change updates a few things, resulting in a much more resilient and reasonable approach to finding the dependencies of link targets, which is relevant for workspaces and for interpreting pnpm trees, and reduces the "load my whole dev folder" antipattern. Prior to this change, in loadActual, if a link target was found in a node_modules folder, then the parent of that node_modules folder would be loaded, along with its children. While this is seems like a reasonable way to handle pnpm trees, it has the following problems: 1. If you are developing a bunch of projects right in a `node_modules` folder, then it would sometimes result in loading ALL of your JS projects in the actualTree! This is rarely what you want, and had some surprising behavior when I encountered it myself. (I develop quite a lot of projects in a node_modules folder for convenience.) 2. If you have a situation where a package is _not_ in a node_modules folder, but it does have some missing dependencies which are in fact resolved by deps already in the tree, it would show up as missing. With this change, the loadActual process will walk up the parent directories of any top nodes, searching for any missing edges. If a folder is found at `${dir}/node_modules/${name}`, then a "dummy" node is created for the parent, such that the parent/fsParent resolution algorithm will find a hit and resolve to the named dep. It will not load any _other_ dependencies in the dummy node, however, and will not walk up higher than the nearest common ancestor of the project root and the link target. Furthermore, this uncovered a bug where setting the parent of a node to a new parent with the same `root` was causing it to unlist itself and all children from the root.meta and root.inventory, but only _relist_ itself, and not its children. This bug was previously not hit in test because the node demonstrating it was within a node_modules folder, and so was fully loaded along with all of its siblings. Lastly, because dummy nodes are never "extraneous" (they're just there as placeholders, they're not "real" anyway), had to update a few shrinkwrap files in fixtures to not get weird results. An upcoming refactor of the Shrinkwrap class will make this incorrect metadata less of a hazard moving forward.

view details

push time in 12 hours

push eventnpm/arborist

claudiahdz

commit sha bb4f1a82436a997b69be8d5879cf2334e81b5896

feat: arborist dedupe()

view details

Ruy Adorno

commit sha cfede2e78b14512aae9f18bfed64e30f1b541bd7

test: dedupe test cases

view details

Ruy Adorno

commit sha 9d01b6deea9c08ced54d15c65e376a40936cddb0

chore: dedupe cleanup - added fixtures for dedupe reify-cases - tweaks and cleanup in lib trimmer/build-ideal-tree Fix: #66 PR-URL: https://github.com/npm/arborist/pull/87 Credit: @ruyadorno, @claudiahdz Close: #87 Reviewed-by: @isaacs, @ruyadorno

view details

push time in 12 hours

PR closed npm/arborist

feat: dedupe

Add arborist.dedupe() - wrapping up the work from @claudiahdz 🥳

+500 -30

0 comment

20 changed files

ruyadorno

pr closed time in 12 hours

issue closednpm/arborist

[FEATURE] arborist.dedupe()

npm dedupe needs to find and de-duplicate dependencies. This should be available as a top-level Arborist function/mixin.

Rough algo sketch:

  • Set this[_preferDedupe] to true
  • Load virtual tree (or, failing that, load actual tree)
  • For each name in tree.inventory.query('name')
    • set nodes to tree.inventory.query('name', name)
    • if nodes.size is 1, continue to next name
    • for each node in nodes
      • for each edge in node.edgesIn
        • add edge.from to this[_depsQueue]
  • set this.idealTree to tree
  • return this.reify()

This will require that we make the _depsQueue and _preferDedupe symbols shared (ie, Symbol.for('_depsQueue') instead of Symbol('_depsQueue').

cc @claudiahdz

closed time in 12 hours

isaacs

Pull request review commentnpm/arborist

feat: dedupe

+// mixin providing prune and dedupe methods++// shared with buildIdealTree+const _depsQueue = Symbol.for('depsQueue')+const _dedupeList = Symbol.for('dedupeList')++module.exports = cls => class Trimmer extends cls {+  constructor (options) {+    super(options)+    this[_dedupeList] = new Set()+  }++  async dedupe () {+    let tree+    try {+      tree = await this.loadVirtual()+    } catch {+      tree = await this.loadActual()+    }+    for (const node of tree.inventory.values()) {+      const nodes = tree.inventory.query('name', node.name)+      if (nodes.size === 1) continue

same style nit as above. Should be:

if (nodes.size === 1)
  continue
ruyadorno

comment created time in 12 hours

Pull request review commentnpm/arborist

feat: dedupe

+// mixin providing prune and dedupe methods++// shared with buildIdealTree+const _depsQueue = Symbol.for('depsQueue')+const _dedupeList = Symbol.for('dedupeList')++module.exports = cls => class Trimmer extends cls {+  constructor (options) {+    super(options)+    this[_dedupeList] = new Set()+  }++  async dedupe () {+    let tree+    try {+      tree = await this.loadVirtual()+    } catch {+      tree = await this.loadActual()+    }+    for (const node of tree.inventory.values()) {+      const nodes = tree.inventory.query('name', node.name)+      if (nodes.size === 1) continue++      for (const node of nodes) {+        this[_dedupeList].add(node.path)+        for (const edge of node.edgesIn) {+          this[_depsQueue].push(edge.from)+        }+      }+    }+    return this.reify({+      preferDedupe: true

Need to include the rest of the options as well, so we can pass other things to dedupe/reify/buildIdealTree.

So, async dedupe (options = {}) { at the start of this function, and then here, do:

return this.reify({
  ...options,
  preferDedupe: true,
})

(Also, trailing comma. Sorry, I'm a pedantic linter today, I guess.)

ruyadorno

comment created time in 12 hours

Pull request review commentnpm/arborist

feat: dedupe

 module.exports = cls => class IdealTreeBuilder extends cls {   // where this dep cannot be placed, and use the one right before that.   // place dep, requested by node, to satisfy edge   [_placeDep] (dep, node, edge, peerEntryEdge = null) {-    if (edge.to && !edge.error && !this[_updateNames].includes(edge.name) &&-        !this[_isVulnerable](edge.to))-      return []+    if (edge.to &&+      !edge.error &&+      !this[_updateNames].includes(edge.name) &&+      !this[_isVulnerable](edge.to) &&+      !this[_dedupeList].has(edge.to.path)) return []

style nit, should be:


if (edge.to &&
    !edge.error &&
    !this[_updateNames].includes(edge.name) &&
    !this[_isVulnerable](edge.to) &&
    !this[_dedupeList].has(edge.to.path))
  return []
  • Double-indent for line-breaks within a ()
  • Always a line-break for single-line conditionals
ruyadorno

comment created time in 12 hours

push eventnpm/arborist

isaacs

commit sha dc61bc9b3a86b7e8611ec8cfec43f50c5f816ccb

Refactor shrinkwrap for correctness and elegance Since the Shrinkwrap class was developed prior to figuring out exactly how it would be used in the various Arborist methods, some things ended up growing in a somewhat inelegant direction. - Nodes were fetching their resolved and integrity values in multiple places. - The Shrinkwrap internal data tried to keep up both the new and old data up to date as the tree changed, but this was not always possible. - Dependencies of link targets were not properly reflected in the legacy shrinkwrap metadata. - The Node class and buildIdealTree methods both had to dive deep into the internals of the YarnLock class, which should be an implementation detail of the Shrinkwrap class. Now, the Shrinkwrap object keeps a reference to the root tree node found at its own path. Rather than try to keep the legacy metadata in sync at all times, we _only_ build up the legacy shrinkwrap data when calling `this.commit()`, by walking the `node.children` and `node.target.children` maps appropriately. Also, `this.yarnLock` is updated on commit(), and a `checkYarnLock()` method is added to return the spec that should be fetched, and update the provided options object with resolved and integrity metadata. All of the logic for setting node metadata from the Shrinkwrap is now done in `Shrinkwrap.add(node)`, making it much simpler and harder to get wrong. This means that: - `yarn.lock` files are respected more fully (including their resolved and integrity expectations) - We don't have cases where both `extraneous` and `dev`/`optional` are set in the lockfile (which is always an error -- extraneous nodes cannot be either dev or optional!) - Link target dependencies are accurately reflected in the legacy shrinkwrap data, rather than creating invalid `../` entries.

view details

push time in 13 hours

PR opened npm/arborist

Isaacs/shrinkwrap refactor

Refactor shrinkwrap for correctness and elegance

Since the Shrinkwrap class was developed prior to figuring out exactly how it would be used in the various Arborist methods, some things ended up growing in a somewhat inelegant direction.

  • Nodes were fetching their resolved and integrity values in multiple places.
  • The Shrinkwrap internal data tried to keep up both the new and old data up to date as the tree changed, but this was not always possible.
  • Dependencies of link targets were not properly reflected in the legacy shrinkwrap metadata.
  • The Node class and buildIdealTree methods both had to dive deep into the internals of the YarnLock class, which should be an implementation detail of the Shrinkwrap class.

Now, the Shrinkwrap object keeps a reference to the root tree node found at its own path. Rather than try to keep the legacy metadata in sync at all times, we only build up the legacy shrinkwrap data when calling this.commit(), by walking the node.children and node.target.children maps appropriately.

Also, this.yarnLock is updated on commit(), and a checkYarnLock() method is added to return the spec that should be fetched, and update the provided options object with resolved and integrity metadata.

All of the logic for setting node metadata from the Shrinkwrap is now done in Shrinkwrap.add(node), making it much simpler and harder to get wrong.

This means that:

  • yarn.lock files are respected more fully (including their resolved and integrity expectations)
  • We don't have cases where both extraneous and dev/optional are set in the lockfile (which is always an error -- extraneous nodes cannot be either dev or optional!)
  • Link target dependencies are accurately reflected in the legacy shrinkwrap data, rather than creating invalid ../ entries.
+17354 -972

0 comment

56 changed files

pr created time in 13 hours

create barnchnpm/arborist

branch : isaacs/shrinkwrap-refactor

created branch time in 13 hours

push eventnpm/arborist

isaacs

commit sha b3d214d9f90b1129b3503a7d4de3a6f4df48b5a6

Rework approach for finding link target deps This change updates a few things, resulting in a much more resilient and reasonable approach to finding the dependencies of link targets, which is relevant for workspaces and for interpreting pnpm trees, and reduces the "load my whole dev folder" antipattern. Prior to this change, in loadActual, if a link target was found in a node_modules folder, then the parent of that node_modules folder would be loaded, along with its children. While this is seems like a reasonable way to handle pnpm trees, it has the following problems: 1. If you are developing a bunch of projects right in a `node_modules` folder, then it would sometimes result in loading ALL of your JS projects in the actualTree! This is rarely what you want, and had some surprising behavior when I encountered it myself. (I develop quite a lot of projects in a node_modules folder for convenience.) 2. If you have a situation where a package is _not_ in a node_modules folder, but it does have some missing dependencies which are in fact resolved by deps already in the tree, it would show up as missing. With this change, the loadActual process will walk up the parent directories of any top nodes, searching for any missing edges. If a folder is found at `${dir}/node_modules/${name}`, then a "dummy" node is created for the parent, such that the parent/fsParent resolution algorithm will find a hit and resolve to the named dep. It will not load any _other_ dependencies in the dummy node, however, and will not walk up higher than the nearest common ancestor of the project root and the link target. Furthermore, this uncovered a bug where setting the parent of a node to a new parent with the same `root` was causing it to unlist itself and all children from the root.meta and root.inventory, but only _relist_ itself, and not its children. This bug was previously not hit in test because the node demonstrating it was within a node_modules folder, and so was fully loaded along with all of its siblings. Lastly, because dummy nodes are never "extraneous" (they're just there as placeholders, they're not "real" anyway), had to update a few shrinkwrap files in fixtures to not get weird results. An upcoming refactor of the Shrinkwrap class will make this incorrect metadata less of a hazard moving forward.

view details

push time in 16 hours

pull request commentnpm/cli

Fix package id in shrinkwrap lifecycle step output

This looks good to me. It's irrelevant in npm v7, but we may as well fix this little thing in the meantime.

Sorry for the delay, we're all pretty focused on getting the next major done. Thanks for your patience.

bz2

comment created time in 17 hours

push eventnpm/arborist

isaacs

commit sha 33e644beee8269cfe02813ca9f05b8a77ba7183a

Rework approach for finding link target deps This change updates a few things, resulting in a much more resilient and reasonable approach to finding the dependencies of link targets, which is relevant for workspaces and for interpreting pnpm trees, and reduces the "load my whole dev folder" antipattern. Prior to this change, in loadActual, if a link target was found in a node_modules folder, then the parent of that node_modules folder would be loaded, along with its children. While this is seems like a reasonable way to handle pnpm trees, it has the following problems: 1. If you are developing a bunch of projects right in a `node_modules` folder, then it would sometimes result in loading ALL of your JS projects in the actualTree! This is rarely what you want, and had some surprising behavior when I encountered it myself. (I develop quite a lot of projects in a node_modules folder for convenience.) 2. If you have a situation where a package is _not_ in a node_modules folder, but it does have some missing dependencies which are in fact resolved by deps already in the tree, it would show up as missing. With this change, the loadActual process will walk up the parent directories of any top nodes, searching for any missing edges. If a folder is found at `${dir}/node_modules/${name}`, then a "dummy" node is created for the parent, such that the parent/fsParent resolution algorithm will find a hit and resolve to the named dep. It will not load any _other_ dependencies in the dummy node, however, and will not walk up higher than the nearest common ancestor of the project root and the link target. Furthermore, this uncovered a bug where setting the parent of a node to a new parent with the same `root` was causing it to unlist itself and all children from the root.meta and root.inventory, but only _relist_ itself, and not its children. This bug was previously not hit in test because the node demonstrating it was within a node_modules folder, and so was fully loaded along with all of its siblings. Lastly, because dummy nodes are never "extraneous" (they're just there as placeholders, they're not "real" anyway), had to update a few shrinkwrap files in fixtures to not get weird results. An upcoming refactor of the Shrinkwrap class will make this incorrect metadata less of a hazard moving forward.

view details

push time in 17 hours

issue closednpm/arborist

[FEATURE] loadActual() should accept {root} option

In loadVirtual we allow setting a {root} option, which uses the supplied node as the root of the tree, instead of creating a new one from the shrinkwrap read at this.path. This is nice when we already have the root node, or if we need to extract a package in a temp directory to read its shrinkwrap, but want to put all the nodes on a node somewhere else (maybe not on disk yet).

We do loadActual in a similar way to read bundleDependencies, but have to manually copy over all the children after loading. It'd be nice to skip that step and just build them out on the node from the start.

closed time in 2 days

isaacs

issue commentnpm/arborist

[FEATURE] loadActual() should accept {root} option

Fixed by #77.

isaacs

comment created time in 2 days

PR opened npm/arborist

Rework approach for finding link target deps

This change updates a few things, resulting in a much more resilient and reasonable approach to finding the dependencies of link targets, which is relevant for workspaces and for interpreting pnpm trees, and reduces the "load my whole dev folder" antipattern.

Prior to this change, in loadActual, if a link target was found in a node_modules folder, then the parent of that node_modules folder would be loaded, along with its children. While this is seems like a reasonable way to handle pnpm trees, it has the following problems:

  1. If you are developing a bunch of projects right in a node_modules folder, then it would sometimes result in loading ALL of your JS projects in the actualTree! This is rarely what you want, and had some surprising behavior when I encountered it myself. (I develop quite a lot of projects in a node_modules folder for convenience.)

  2. If you have a situation where a package is not in a node_modules folder, but it does have some missing dependencies which are in fact resolved by deps already in the tree, it would show up as missing.

With this change, the loadActual process will walk up the parent directories of any top nodes, searching for any missing edges. If a folder is found at ${dir}/node_modules/${name}, then a "dummy" node is created for the parent, such that the parent/fsParent resolution algorithm will find a hit and resolve to the named dep.

It will not load any other dependencies in the dummy node, however, and will not walk up higher than the nearest common ancestor of the project root and the link target.

Furthermore, this uncovered a bug where setting the parent of a node to a new parent with the same root was causing it to unlist itself and all children from the root.meta and root.inventory, but only relist itself, and not its children. This bug was previously not hit in test because the node demonstrating it was within a node_modules folder, and so was fully loaded along with all of its siblings.

What / Why

<!-- Describe the request in detail -->

n/a

References

<!-- Examples

  • Related to #0
  • Depends on #0
  • Blocked by #0
  • Closes #0 -->
  • n/a
+1533 -490

0 comment

41 changed files

pr created time in 2 days

created tagisaacs/common-ancestor-path

tagv1.0.1

created time in 2 days

push eventisaacs/common-ancestor-path

isaacs

commit sha 1f3c63da7e543ef31130a7fe77188ff9fcf7236a

oops, forgot to include the thing in the package

view details

isaacs

commit sha 0bf875bd22a01c40364a8f9fb185c78cf27d0fb0

1.0.1

view details

push time in 2 days

created tagisaacs/common-ancestor-path

tagv1.0.0

created time in 2 days

create barnchisaacs/common-ancestor-path

branch : master

created branch time in 2 days

created repositoryisaacs/common-ancestor-path

created time in 2 days

PR closed isaacs/node-mkdirp

[Security Vuln.] Upgrade tap to 14.10.7

Issue: #11

Description of changes

This upgrades tap to address a vulnerability found in minimist. Despite being a patch bump I've run tests to double check and they're also passing.

If there's anything else I can do to help please let me know.

NPM audit after upgrade

{
  "actions": [],
  "advisories": {},
  "muted": [],
  "metadata": {
    "vulnerabilities": {
      "info": 0,
      "low": 0,
      "moderate": 0,
      "high": 0,
      "critical": 0
    },
    "dependencies": 0,
    "devDependencies": 1318,
    "optionalDependencies": 1,
    "totalDependencies": 1318
  },
  "runId": "2a9293ad-0b62-406d-8902-9886990c5a20"
}

NPM audit before this upgrade

{
  "actions": [
    {
      "action": "update",
      "resolves": [
        {
          "id": 1179,
          "path": "tap>coveralls>minimist",
          "dev": true,
          "optional": false,
          "bundled": false
        }
      ],
      "module": "minimist",
      "target": "1.2.5",
      "depth": 3
    },
    {
      "action": "review",
      "module": "minimist",
      "resolves": [
        {
          "id": 1179,
          "path": "tap>import-jsx>@babel/core>json5>minimist",
          "dev": true,
          "optional": false,
          "bundled": true
        },
        {
          "id": 1179,
          "path": "tap>treport>import-jsx>@babel/core>json5>minimist",
          "dev": true,
          "optional": false,
          "bundled": true
        },
        {
          "id": 1179,
          "path": "tap>mkdirp>minimist",
          "dev": true,
          "optional": false,
          "bundled": false
        },
        {
          "id": 1179,
          "path": "tap>nyc>spawn-wrap>mkdirp>minimist",
          "dev": true,
          "optional": false,
          "bundled": false
        }
      ]
    }
  ],
  "advisories": {
    "1179": {
      "findings": [
        {
          "version": "1.2.0",
          "paths": [
            "tap>coveralls>minimist"
          ]
        },
        {
          "version": "1.2.0",
          "paths": [
            "tap>import-jsx>@babel/core>json5>minimist",
            "tap>treport>import-jsx>@babel/core>json5>minimist"
          ]
        },
        {
          "version": "0.0.8",
          "paths": [
            "tap>mkdirp>minimist",
            "tap>nyc>spawn-wrap>mkdirp>minimist"
          ]
        }
      ],
      "id": 1179,
      "created": "2019-09-23T15:01:43.049Z",
      "updated": "2020-03-18T19:41:45.921Z",
      "deleted": null,
      "title": "Prototype Pollution",
      "found_by": {
        "link": "https://www.checkmarx.com/resources/blog/",
        "name": "Checkmarx Research Team",
        "email": ""
      },
      "reported_by": {
        "link": "https://www.checkmarx.com/resources/blog/",
        "name": "Checkmarx Research Team",
        "email": ""
      },
      "module_name": "minimist",
      "cves": [],
      "vulnerable_versions": "<0.2.1 || >=1.0.0 <1.2.3",
      "patched_versions": ">=0.2.1 <1.0.0 || >=1.2.3",
      "overview": "Affected versions of `minimist` are vulnerable to prototype pollution. Arguments are not properly sanitized, allowing an attacker to modify the prototype of `Object`, causing the addition or modification of an existing property that will exist on all objects.  \nParsing the argument `--__proto__.y=Polluted` adds a `y` property with value `Polluted` to all objects. The argument `--__proto__=Polluted` raises and uncaught error and crashes the application.  \nThis is exploitable if attackers have control over the arguments being passed to `minimist`.\n",
      "recommendation": "Upgrade to versions 0.2.1, 1.2.3 or later.",
      "references": "- [GitHub commit 1](https://github.com/substack/minimist/commit/4cf1354839cb972e38496d35e12f806eea92c11f#diff-a1e0ee62c91705696ddb71aa30ad4f95)\n- [GitHub commit 2](https://github.com/substack/minimist/commit/63e7ed05aa4b1889ec2f3b196426db4500cbda94)",
      "access": "public",
      "severity": "low",
      "cwe": "CWE-471",
      "metadata": {
        "module_type": "",
        "exploitability": 1,
        "affected_components": ""
      },
      "url": "https://npmjs.com/advisories/1179"
    }
  },
  "muted": [],
  "metadata": {
    "vulnerabilities": {
      "info": 0,
      "low": 5,
      "moderate": 0,
      "high": 0,
      "critical": 0
    },
    "dependencies": 0,
    "devDependencies": 1316,
    "optionalDependencies": 1,
    "totalDependencies": 1316
  },
  "runId": "a5407ca8-8235-48f8-8329-b99e8d1f3cc3"
}

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

+155 -177

0 comment

2 changed files

heitorlessa

pr closed time in 4 days

issue commentisaacs/node-mkdirp

mkdirp failed to create dir if running on pm2 and create di on /mnt share directory (network dir)

What version of mkdirp are you using? What error are you getting?

rusliabdulgani

comment created time in 4 days

issue closedisaacs/node-mkdirp

mkdirp error

code: var mkdirp = require('mkdirp');

mkdirp('/tmp/foo/bar/baz', function (err) { if (err){ console.log(e); }else{ console.log('we are good.') } });

error: TypeError: invalid options argument at optsArg (D:\web\node_modules\mkdirp\lib\opts-arg.js:13:11) at mkdirp (D:\web\node_modules\mkdirp\index.js:11:10)

closed time in 4 days

mxnxsh

issue commentisaacs/node-mkdirp

mkdirp error

That code is for mkdirp 0.x, and you're using 1.0. See the docs for current usage.

mxnxsh

comment created time in 4 days

issue openednpm/arborist

link target metadeps not saved in legacy shrinkwrap dependencies properly

Given a folder structure like this:

.
├── node_modules
│  └── a -> ../once
├── once
│  ├── node_modules
│  │  └── wrappy
│  │     └── package.json
│  └── package.json
├── package-lock.json
└── package.json

npm v6 produces a shrinkwrap like this:

{
  "requires": true,
  "lockfileVersion": 1,
  "dependencies": {
    "a": {
      "version": "file:once",
      "requires": {
        "wrappy": "1"
      },
      "dependencies": {
        "wrappy": {
          "version": "1.0.2",
          "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
          "integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8="
        }
      }
    }
  }
}

But arborist is doing this in the legacy shrinkwrap section:

{
  "dependencies": {
    "a": {
      "version": "file:once"
    },
    "once": {
      "dependencies": {
        "wrappy": {
          "version": "1.0.2",
          "resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
          "integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8="
        }
      }
    }
  }
}

That's wrong.

It seems like the thing to do here is start from the root Node object, and walk the children (using target.children for links) and build the object that way.

The Shrinkwrap class needs to keep a reference to the tree representing its root node (replacing if we ever give it a new node at the '' location), and should have a Map of location -> metadata. If there are nodes that don't end up being actualized, we can still write back the metadata for those locations. (So, for example, a reify() that omits dev deps will still have them in the lockfile). Then we can also trivially write the yarn.lock on save(), solving #82.

The reason why it didn't do this initially was a notion that Shrinkwrap and Node should be separate concerns, and shouldn't know too much about each other. However, in practice, they're all up in each other business anyway, and we're doing a lot of translating path strings into object paths and so on. Since the two classes really can't live apart from one another, we may as well have a more straightforward way of having them interact.

created time in 4 days

Pull request review commentnpm/arborist

fix: Handle tarball dependencies properly

 Node {       "ghtgz": Object {         "extraneous": true,         "integrity": "sha512-yowslMd9y/lGBCDVO0RwZoXRK5X0zMsf6XECM6DdeqN7qwVnFQ6IAwJai7BD4mVe1xOdWWqWNkuzyuStvSBnHw==",-        "resolved": "https://codeload.github.com/isaacs/abbrev-js/tar.gz/a9ee72ebc8fe3975f1b0c7aeb3a8f2a806a432eb",-        "version": "npm:abbrev@1.1.1",+        "version": "https://codeload.github.com/isaacs/abbrev-js/tar.gz/a9ee72ebc8fe3975f1b0c7aeb3a8f2a806a432eb",

Interesting side effect of the changes in this: there was previously a bug where a remote tarball dep would be saved incorrectly as an alias dep.

isaacs

comment created time in 5 days

Pull request review commentnpm/arborist

fix: Handle tarball dependencies properly

 Node { } ` +exports[`test/arborist/build-ideal-tree.js TAP tarball deps with transitive tarball deps > expect resolving Promise 1`] = `+Node {+  "children": Map {+    "@isaacs/testing-file-transitive-a" => Node {+      "edgesIn": Set {+        Edge {+          "from": "",+          "name": "@isaacs/testing-file-transitive-a",+          "spec": "file:testing-file-transitive-a/isaacs-testing-file-transitive-a-1.0.0.tgz",+          "type": "prod",+        },+      },+      "edgesOut": Map {+        "@isaacs/testing-file-transitive-b" => Edge {+          "name": "@isaacs/testing-file-transitive-b",+          "spec": "file:../testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz",+          "to": "node_modules/@isaacs/testing-file-transitive-b",+          "type": "prod",+        },+        "@isaacs/testing-file-transitive-c" => Edge {+          "name": "@isaacs/testing-file-transitive-c",+          "spec": "file:./c/isaacs-testing-file-transitive-c-1.0.0.tgz",+          "to": "node_modules/@isaacs/testing-file-transitive-c",+          "type": "prod",+        },+      },+      "location": "node_modules/@isaacs/testing-file-transitive-a",+      "name": "@isaacs/testing-file-transitive-a",+      "resolved": "file:{CWD}/test/fixtures/tarball-dependencies/testing-file-transitive-a/isaacs-testing-file-transitive-a-1.0.0.tgz",+    },+    "@isaacs/testing-file-transitive-b" => Node {+      "edgesIn": Set {+        Edge {+          "from": "node_modules/@isaacs/testing-file-transitive-a",+          "name": "@isaacs/testing-file-transitive-b",+          "spec": "file:../testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz",+          "type": "prod",+        },+      },+      "location": "node_modules/@isaacs/testing-file-transitive-b",+      "name": "@isaacs/testing-file-transitive-b",+      "resolved": "file:{CWD}/test/fixtures/tarball-dependencies/testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz",+    },+    "@isaacs/testing-file-transitive-c" => Node {+      "edgesIn": Set {+        Edge {+          "from": "node_modules/@isaacs/testing-file-transitive-a",+          "name": "@isaacs/testing-file-transitive-c",+          "spec": "file:./c/isaacs-testing-file-transitive-c-1.0.0.tgz",+          "type": "prod",+        },+      },+      "location": "node_modules/@isaacs/testing-file-transitive-c",+      "name": "@isaacs/testing-file-transitive-c",+      "resolved": "file:{CWD}/test/fixtures/tarball-dependencies/testing-file-transitive-a/c/isaacs-testing-file-transitive-c-1.0.0.tgz",+    },+  },+  "edgesOut": Map {+    "@isaacs/testing-file-transitive-a" => Edge {+      "name": "@isaacs/testing-file-transitive-a",+      "spec": "file:testing-file-transitive-a/isaacs-testing-file-transitive-a-1.0.0.tgz",+      "to": "node_modules/@isaacs/testing-file-transitive-a",+      "type": "prod",+    },+  },+  "location": "",+  "name": "tarball-dependencies",+  "resolved": null,+}+`+

That is, the root depends on file:testing-file-transitive-a/isaacs-testing-file-transitive-a-1.0.0.tgz, and then a depends on the file at ./testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz, which from a's pov is ../testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz.

isaacs

comment created time in 5 days

Pull request review commentnpm/arborist

fix: Handle tarball dependencies properly

 Node { } ` +exports[`test/arborist/build-ideal-tree.js TAP tarball deps with transitive tarball deps > expect resolving Promise 1`] = `+Node {+  "children": Map {+    "@isaacs/testing-file-transitive-a" => Node {+      "edgesIn": Set {+        Edge {+          "from": "",+          "name": "@isaacs/testing-file-transitive-a",+          "spec": "file:testing-file-transitive-a/isaacs-testing-file-transitive-a-1.0.0.tgz",+          "type": "prod",+        },+      },+      "edgesOut": Map {+        "@isaacs/testing-file-transitive-b" => Edge {+          "name": "@isaacs/testing-file-transitive-b",+          "spec": "file:../testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz",+          "to": "node_modules/@isaacs/testing-file-transitive-b",+          "type": "prod",+        },+        "@isaacs/testing-file-transitive-c" => Edge {+          "name": "@isaacs/testing-file-transitive-c",+          "spec": "file:./c/isaacs-testing-file-transitive-c-1.0.0.tgz",+          "to": "node_modules/@isaacs/testing-file-transitive-c",+          "type": "prod",+        },+      },+      "location": "node_modules/@isaacs/testing-file-transitive-a",+      "name": "@isaacs/testing-file-transitive-a",+      "resolved": "file:{CWD}/test/fixtures/tarball-dependencies/testing-file-transitive-a/isaacs-testing-file-transitive-a-1.0.0.tgz",+    },+    "@isaacs/testing-file-transitive-b" => Node {+      "edgesIn": Set {+        Edge {+          "from": "node_modules/@isaacs/testing-file-transitive-a",+          "name": "@isaacs/testing-file-transitive-b",+          "spec": "file:../testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz",+          "type": "prod",+        },+      },+      "location": "node_modules/@isaacs/testing-file-transitive-b",+      "name": "@isaacs/testing-file-transitive-b",+      "resolved": "file:{CWD}/test/fixtures/tarball-dependencies/testing-file-transitive-b/isaacs-testing-file-transitive-b-1.0.0.tgz",+    },+    "@isaacs/testing-file-transitive-c" => Node {+      "edgesIn": Set {+        Edge {+          "from": "node_modules/@isaacs/testing-file-transitive-a",+          "name": "@isaacs/testing-file-transitive-c",+          "spec": "file:./c/isaacs-testing-file-transitive-c-1.0.0.tgz",+          "type": "prod",+        },+      },+      "location": "node_modules/@isaacs/testing-file-transitive-c",+      "name": "@isaacs/testing-file-transitive-c",+      "resolved": "file:{CWD}/test/fixtures/tarball-dependencies/testing-file-transitive-a/c/isaacs-testing-file-transitive-c-1.0.0.tgz",+    },+  },+  "edgesOut": Map {+    "@isaacs/testing-file-transitive-a" => Edge {+      "name": "@isaacs/testing-file-transitive-a",+      "spec": "file:testing-file-transitive-a/isaacs-testing-file-transitive-a-1.0.0.tgz",+      "to": "node_modules/@isaacs/testing-file-transitive-a",+      "type": "prod",+    },+  },+  "location": "",+  "name": "tarball-dependencies",+  "resolved": null,+}+`+

The reason for the ../ there is that the edges are resolved relative to the folder where the tarball is found.

So, the root has:

+-- testing-transitive-a/isaacs-testing-transitive-a-1.0.0.tgz
+-- testing-transitive-b/isaacs-testing-transitive-b-1.0.0.tgz

The testing-transitive-a package's package.json has a dep on ../testing-transitive-b/isaacs-testing-transitive-b-1.0.0.tgz. Ie, a reference from the a package to the b package which is its peer.

When we resolve file spec edges on file-resolved packages, the resulting target is the resolution against the dirname of the requiring module's resolved value.

isaacs

comment created time in 5 days

push eventnpm/arborist

isaacs

commit sha ffe57201350c79eedd57fbec4f4a09670f72ddf1

remove unused load-bundle-deps module PR-URL: https://github.com/npm/arborist/pull/77 Credit: @isaacs Close: #77 Reviewed-by: @ruyadorno

view details

isaacs

commit sha ce51fc962efdcae90c8deda5f1248e041abefd32

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46 PR-URL: https://github.com/npm/arborist/pull/78 Credit: @isaacs Close: #78 Reviewed-by: @ruyadorno

view details

isaacs

commit sha 6ea73de40aaa362cc32cc6911e58e459ea27c8bf

test: add proper caching where it was missing

view details

isaacs

commit sha d4ab8b3c273f29e555bfc2a0ac5fb59d8aaa2ef5

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46 PR-URL: https://github.com/npm/arborist/pull/80 Credit: @isaacs Close: #80 Reviewed-by: @ruyadorno

view details

isaacs

commit sha 7e92f6cea0e9ea8cd90aa71dffe2a17d86486a17

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha 94dac47569091f87a99b81f83257a1805664bd61

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

isaacs

commit sha 948e6387570e00e694f5e973d8bd3582a0dcab55

add Arborist.rebuild() This refactors reify() to move bin linking and script running into the rebuild() method, and exposes it for use in 'npm rebuild'

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha ffe57201350c79eedd57fbec4f4a09670f72ddf1

remove unused load-bundle-deps module PR-URL: https://github.com/npm/arborist/pull/77 Credit: @isaacs Close: #77 Reviewed-by: @ruyadorno

view details

isaacs

commit sha ce51fc962efdcae90c8deda5f1248e041abefd32

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46 PR-URL: https://github.com/npm/arborist/pull/78 Credit: @isaacs Close: #78 Reviewed-by: @ruyadorno

view details

isaacs

commit sha 6ea73de40aaa362cc32cc6911e58e459ea27c8bf

test: add proper caching where it was missing

view details

isaacs

commit sha d4ab8b3c273f29e555bfc2a0ac5fb59d8aaa2ef5

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46 PR-URL: https://github.com/npm/arborist/pull/80 Credit: @isaacs Close: #80 Reviewed-by: @ruyadorno

view details

isaacs

commit sha 7e92f6cea0e9ea8cd90aa71dffe2a17d86486a17

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha 94dac47569091f87a99b81f83257a1805664bd61

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

push time in 5 days

issue commentnpm/arborist

Interesting change from npm v6: install with no shrinkwrap does a full update

One hazard here is that reify calls loadActual and buildIdealTree in parallel, so we'll need to make sure the actual tree is actually done before proceeding. Ie, make loadActual parallel-safe by stashing a loading promise, and resolving it at the end.

isaacs

comment created time in 5 days

issue commentnpm/cli

[FEATURE] new install protocol file+pack

Also, spelling of the protocol needs to be bikeshedded. I don't know off the top of my head any reason to prefer file+pack: vs pack: vs pack+file: etc., but I know we do test the resolved string against /^file:/ in a few places to see whether it's a local dep, so all those code paths will have to be examined.

RecuencoJones

comment created time in 5 days

issue commentnpm/cli

[FEATURE] new install protocol file+pack

This should be an rfc for sure. It's a good idea, and one that might make my personal day to day work a bit easier. If so, I'd be eager to get it implemented 😂

Right now the workaround is to pack the dir, and then install the packed tarball. That of course works, but it's an extra step. The advantage there is that the installed dep only changes when the artifact is re-generated. The disadvantage, obviously, is the extra step.

Some other thoughts:

  • It seems like at least part of the need here stems from the fact that you want the prepare script to be run in the linked dep. Could we get part of the way with a flag to just tell npm to do that? Ie, npm install --rebuild-links or something? That flag would also be respected by default by the rebuild command, so you could do npm rebuild --rebuild-links when you know something's changed, and even set up a watcher to do that while in development. (This would be considerably faster than creating and unpacking a tarball.)

  • If the bytes don't change, the integrity won't change. (As of npm v7, the integrity of the gzipped tarball will also be consistent across operating systems, because we strip out the annoying useless hard-coded gzip OS header.) We may need to have a slightly different strategy for reifying these deps though. A matching integrity in the lockfile and a newly generated artifact will show us that nothing has changed, but a changed integrity means that we need to replace it. So, it's less an expectation, per se, and more a way to just track what we already have and avoid extra work, maybe? I would be surprised if I change something in my locally packed dep, and then the install fails with an EINTEGRITY.

  • For publishing and sharing, I see this as roughly equivalent to tarballs or symlink deps, which we do allow you to publish with today, and has for a long time. It's not super portable, of course, but it doesn't seem to cause serious problems, and people using this feature tend to understand the impact. (Or if they don't, they learn quickly when their users complain.)

RecuencoJones

comment created time in 5 days

pull request commentnpm/rfcs

RFC: Add `npm-app-id` HTTP header

@jlstephens89 You can also watch the full discussion, if you wanna feel like you were there :) https://www.youtube.com/watch?v=W6XbJ5e3xrA

Mykyta

comment created time in 5 days

push eventnpm/arborist

isaacs

commit sha 4f8c6c9541bb4266365a67f4f213e853a88d23de

Fully update metadata/location when moving roots There were some cases, especially when moving fsChildren from one root to another, where the node.location would not be updated in the correct sequence, resulting in the metadata being incorrect or missing, and node.location being out of sync with node.path and node.root.path. Discovered while adding functionality for #72

view details

isaacs

commit sha 92176417f6f22b1998833e306c5104310a293bcc

Add loadActual({ root: Node }) to compose trees Implementation for #72

view details

isaacs

commit sha 89fb80fa9427b6ac112e09e41dafc106c8dc4408

loadActual: cleanup code using async/await

view details

isaacs

commit sha 9b979840479216c874a3967659f692823eb38b49

calcDepFlags: add 'resetRoot' flag to tell it to not do that When we call loadActual to load bundle deps, we do NOT want to reset all the dev/optional/etc flags.

view details

isaacs

commit sha 5ee0d16502d0e9bd35985d0d2b075f6158d1ca28

loadActual: add transplantFilter method to filter what gets transplanted

view details

isaacs

commit sha 32747a258f46fbf1835fa16b381b5f2b31c4f263

reify: replace load-bundle-deps module with loadActual({root}) This makes it so that we get proper dep flags in bundled deps! Win!

view details

isaacs

commit sha 5fac0af5a7d1f821c07c610e5a81aa4d67afc9b2

buildIdeal: replace load-bundle-deps module with loadActual({root})

view details

isaacs

commit sha 213800882db2c48dcaa6f335ce61b90b512d932e

remove unused load-bundle-deps module

view details

isaacs

commit sha 7be34fa9832cb18da18430a3c4354962156a36ee

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

isaacs

commit sha f736bae9ba1b57ca27574fdf30d3dc5d2665f2a5

test: add proper caching where it was missing

view details

isaacs

commit sha c5118086431fe7492f6c143acb2cfd8866af008f

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46

view details

isaacs

commit sha d0de5de054077837eeefc97247eb5c0a739cfd47

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha 10a88c5c389130d6f9df534644b9b68c0e081a53

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

isaacs

commit sha 33ec372b230c6ed45f619943d9c6adec4b0b61bb

add Arborist.rebuild() This refactors reify() to move bin linking and script running into the rebuild() method, and exposes it for use in 'npm rebuild'

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha 4f8c6c9541bb4266365a67f4f213e853a88d23de

Fully update metadata/location when moving roots There were some cases, especially when moving fsChildren from one root to another, where the node.location would not be updated in the correct sequence, resulting in the metadata being incorrect or missing, and node.location being out of sync with node.path and node.root.path. Discovered while adding functionality for #72

view details

isaacs

commit sha 92176417f6f22b1998833e306c5104310a293bcc

Add loadActual({ root: Node }) to compose trees Implementation for #72

view details

isaacs

commit sha 89fb80fa9427b6ac112e09e41dafc106c8dc4408

loadActual: cleanup code using async/await

view details

isaacs

commit sha 9b979840479216c874a3967659f692823eb38b49

calcDepFlags: add 'resetRoot' flag to tell it to not do that When we call loadActual to load bundle deps, we do NOT want to reset all the dev/optional/etc flags.

view details

isaacs

commit sha 5ee0d16502d0e9bd35985d0d2b075f6158d1ca28

loadActual: add transplantFilter method to filter what gets transplanted

view details

isaacs

commit sha 32747a258f46fbf1835fa16b381b5f2b31c4f263

reify: replace load-bundle-deps module with loadActual({root}) This makes it so that we get proper dep flags in bundled deps! Win!

view details

isaacs

commit sha 5fac0af5a7d1f821c07c610e5a81aa4d67afc9b2

buildIdeal: replace load-bundle-deps module with loadActual({root})

view details

isaacs

commit sha 213800882db2c48dcaa6f335ce61b90b512d932e

remove unused load-bundle-deps module

view details

isaacs

commit sha 7be34fa9832cb18da18430a3c4354962156a36ee

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

isaacs

commit sha f736bae9ba1b57ca27574fdf30d3dc5d2665f2a5

test: add proper caching where it was missing

view details

isaacs

commit sha c5118086431fe7492f6c143acb2cfd8866af008f

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46

view details

isaacs

commit sha d0de5de054077837eeefc97247eb5c0a739cfd47

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha 10a88c5c389130d6f9df534644b9b68c0e081a53

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha 22439ce97e1005bfa2079ff36c29083a57010b31

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

isaacs

commit sha 9043c73ff3101b203698cbcf3bd000da9ef6c670

test: add proper caching where it was missing

view details

isaacs

commit sha 8f514c24c9ddc1e2edd4800d9dfcbe56b6839a8c

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46

view details

isaacs

commit sha 7cb3777c90b1190e895ed924a7298760eb5b087d

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha 8c658709c274e5ec85d0488e93d3d3ace03e2d81

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha 22439ce97e1005bfa2079ff36c29083a57010b31

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

isaacs

commit sha 9043c73ff3101b203698cbcf3bd000da9ef6c670

test: add proper caching where it was missing

view details

isaacs

commit sha 8f514c24c9ddc1e2edd4800d9dfcbe56b6839a8c

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46

view details

isaacs

commit sha 7cb3777c90b1190e895ed924a7298760eb5b087d

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha 8c658709c274e5ec85d0488e93d3d3ace03e2d81

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

isaacs

commit sha e663f1776f7ebae0b3fec88a5d6b4cbbce6fd898

add Arborist.rebuild() This refactors reify() to move bin linking and script running into the rebuild() method, and exposes it for use in 'npm rebuild'

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha 4f8c6c9541bb4266365a67f4f213e853a88d23de

Fully update metadata/location when moving roots There were some cases, especially when moving fsChildren from one root to another, where the node.location would not be updated in the correct sequence, resulting in the metadata being incorrect or missing, and node.location being out of sync with node.path and node.root.path. Discovered while adding functionality for #72

view details

isaacs

commit sha 92176417f6f22b1998833e306c5104310a293bcc

Add loadActual({ root: Node }) to compose trees Implementation for #72

view details

isaacs

commit sha 89fb80fa9427b6ac112e09e41dafc106c8dc4408

loadActual: cleanup code using async/await

view details

isaacs

commit sha 9b979840479216c874a3967659f692823eb38b49

calcDepFlags: add 'resetRoot' flag to tell it to not do that When we call loadActual to load bundle deps, we do NOT want to reset all the dev/optional/etc flags.

view details

isaacs

commit sha 5ee0d16502d0e9bd35985d0d2b075f6158d1ca28

loadActual: add transplantFilter method to filter what gets transplanted

view details

isaacs

commit sha 32747a258f46fbf1835fa16b381b5f2b31c4f263

reify: replace load-bundle-deps module with loadActual({root}) This makes it so that we get proper dep flags in bundled deps! Win!

view details

isaacs

commit sha 5fac0af5a7d1f821c07c610e5a81aa4d67afc9b2

buildIdeal: replace load-bundle-deps module with loadActual({root})

view details

isaacs

commit sha 213800882db2c48dcaa6f335ce61b90b512d932e

remove unused load-bundle-deps module

view details

isaacs

commit sha 7be34fa9832cb18da18430a3c4354962156a36ee

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

isaacs

commit sha f736bae9ba1b57ca27574fdf30d3dc5d2665f2a5

test: add proper caching where it was missing

view details

isaacs

commit sha c5118086431fe7492f6c143acb2cfd8866af008f

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46

view details

isaacs

commit sha 6f1d3b76764341116dbe0a66eb6c2acbbdcb5cf2

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha d1bb4f3147db19335857c80867f62ea447aac704

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

isaacs

commit sha 30229affabd3b351d3f4fffd9cb99106d06377ec

add Arborist.rebuild() This refactors reify() to move bin linking and script running into the rebuild() method, and exposes it for use in 'npm rebuild'

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha 4f8c6c9541bb4266365a67f4f213e853a88d23de

Fully update metadata/location when moving roots There were some cases, especially when moving fsChildren from one root to another, where the node.location would not be updated in the correct sequence, resulting in the metadata being incorrect or missing, and node.location being out of sync with node.path and node.root.path. Discovered while adding functionality for #72

view details

isaacs

commit sha 92176417f6f22b1998833e306c5104310a293bcc

Add loadActual({ root: Node }) to compose trees Implementation for #72

view details

isaacs

commit sha 89fb80fa9427b6ac112e09e41dafc106c8dc4408

loadActual: cleanup code using async/await

view details

isaacs

commit sha 9b979840479216c874a3967659f692823eb38b49

calcDepFlags: add 'resetRoot' flag to tell it to not do that When we call loadActual to load bundle deps, we do NOT want to reset all the dev/optional/etc flags.

view details

isaacs

commit sha 5ee0d16502d0e9bd35985d0d2b075f6158d1ca28

loadActual: add transplantFilter method to filter what gets transplanted

view details

isaacs

commit sha 32747a258f46fbf1835fa16b381b5f2b31c4f263

reify: replace load-bundle-deps module with loadActual({root}) This makes it so that we get proper dep flags in bundled deps! Win!

view details

isaacs

commit sha 5fac0af5a7d1f821c07c610e5a81aa4d67afc9b2

buildIdeal: replace load-bundle-deps module with loadActual({root})

view details

isaacs

commit sha 213800882db2c48dcaa6f335ce61b90b512d932e

remove unused load-bundle-deps module

view details

isaacs

commit sha 7be34fa9832cb18da18430a3c4354962156a36ee

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

isaacs

commit sha f736bae9ba1b57ca27574fdf30d3dc5d2665f2a5

test: add proper caching where it was missing

view details

isaacs

commit sha c5118086431fe7492f6c143acb2cfd8866af008f

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46

view details

isaacs

commit sha 6f1d3b76764341116dbe0a66eb6c2acbbdcb5cf2

Refactor the Arborist class mixin stack There's no reason why these mixins have to stack on one another in the definition or extend one another. We only ever use the Arborist class as a fully-assembled thing, so it's fine to just throw them all in a list and call it a day.

view details

isaacs

commit sha d1bb4f3147db19335857c80867f62ea447aac704

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha 4f8c6c9541bb4266365a67f4f213e853a88d23de

Fully update metadata/location when moving roots There were some cases, especially when moving fsChildren from one root to another, where the node.location would not be updated in the correct sequence, resulting in the metadata being incorrect or missing, and node.location being out of sync with node.path and node.root.path. Discovered while adding functionality for #72

view details

isaacs

commit sha 92176417f6f22b1998833e306c5104310a293bcc

Add loadActual({ root: Node }) to compose trees Implementation for #72

view details

isaacs

commit sha 89fb80fa9427b6ac112e09e41dafc106c8dc4408

loadActual: cleanup code using async/await

view details

isaacs

commit sha 9b979840479216c874a3967659f692823eb38b49

calcDepFlags: add 'resetRoot' flag to tell it to not do that When we call loadActual to load bundle deps, we do NOT want to reset all the dev/optional/etc flags.

view details

isaacs

commit sha 5ee0d16502d0e9bd35985d0d2b075f6158d1ca28

loadActual: add transplantFilter method to filter what gets transplanted

view details

isaacs

commit sha 32747a258f46fbf1835fa16b381b5f2b31c4f263

reify: replace load-bundle-deps module with loadActual({root}) This makes it so that we get proper dep flags in bundled deps! Win!

view details

isaacs

commit sha 5fac0af5a7d1f821c07c610e5a81aa4d67afc9b2

buildIdeal: replace load-bundle-deps module with loadActual({root})

view details

isaacs

commit sha 213800882db2c48dcaa6f335ce61b90b512d932e

remove unused load-bundle-deps module

view details

isaacs

commit sha 7be34fa9832cb18da18430a3c4354962156a36ee

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

isaacs

commit sha f736bae9ba1b57ca27574fdf30d3dc5d2665f2a5

test: add proper caching where it was missing

view details

isaacs

commit sha c5118086431fe7492f6c143acb2cfd8866af008f

Support rebuildBundle: false Re: https://github.com/npm/arborist/issues/46

view details

push time in 5 days

push eventnpm/arborist

isaacs

commit sha 4f8c6c9541bb4266365a67f4f213e853a88d23de

Fully update metadata/location when moving roots There were some cases, especially when moving fsChildren from one root to another, where the node.location would not be updated in the correct sequence, resulting in the metadata being incorrect or missing, and node.location being out of sync with node.path and node.root.path. Discovered while adding functionality for #72

view details

isaacs

commit sha 92176417f6f22b1998833e306c5104310a293bcc

Add loadActual({ root: Node }) to compose trees Implementation for #72

view details

isaacs

commit sha 89fb80fa9427b6ac112e09e41dafc106c8dc4408

loadActual: cleanup code using async/await

view details

isaacs

commit sha 9b979840479216c874a3967659f692823eb38b49

calcDepFlags: add 'resetRoot' flag to tell it to not do that When we call loadActual to load bundle deps, we do NOT want to reset all the dev/optional/etc flags.

view details

isaacs

commit sha 5ee0d16502d0e9bd35985d0d2b075f6158d1ca28

loadActual: add transplantFilter method to filter what gets transplanted

view details

isaacs

commit sha 32747a258f46fbf1835fa16b381b5f2b31c4f263

reify: replace load-bundle-deps module with loadActual({root}) This makes it so that we get proper dep flags in bundled deps! Win!

view details

isaacs

commit sha 5fac0af5a7d1f821c07c610e5a81aa4d67afc9b2

buildIdeal: replace load-bundle-deps module with loadActual({root})

view details

isaacs

commit sha 213800882db2c48dcaa6f335ce61b90b512d932e

remove unused load-bundle-deps module

view details

isaacs

commit sha 7be34fa9832cb18da18430a3c4354962156a36ee

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

push time in 5 days

push eventnpm/arborist

Ruy Adorno

commit sha 983e7517bc427ccac9d55c912fe12444936ed830

fix: avoid throwing type errors on inventory.add Trying to load mock malformed testing data from the cli resulted in an unexpected TypeError. The issue can be reproduced by having a falsy funding property in a package.json, as seen here: https://github.com/npm/cli/blob/abdf52879fcf0e0f534ad977931f6935f5d1dce3/test/tap/fund.js#L81 This commit fixes the problem by avoiding trying to retrieve the `type` and `url` properties when having a `null` object reference. PR-URL: https://github.com/npm/arborist/pull/76 Credit: @ruyadorno Close: #76 Reviewed-by: @isaacs

view details

isaacs

commit sha 4f8c6c9541bb4266365a67f4f213e853a88d23de

Fully update metadata/location when moving roots There were some cases, especially when moving fsChildren from one root to another, where the node.location would not be updated in the correct sequence, resulting in the metadata being incorrect or missing, and node.location being out of sync with node.path and node.root.path. Discovered while adding functionality for #72

view details

isaacs

commit sha 92176417f6f22b1998833e306c5104310a293bcc

Add loadActual({ root: Node }) to compose trees Implementation for #72

view details

isaacs

commit sha 89fb80fa9427b6ac112e09e41dafc106c8dc4408

loadActual: cleanup code using async/await

view details

isaacs

commit sha 9b979840479216c874a3967659f692823eb38b49

calcDepFlags: add 'resetRoot' flag to tell it to not do that When we call loadActual to load bundle deps, we do NOT want to reset all the dev/optional/etc flags.

view details

isaacs

commit sha 5ee0d16502d0e9bd35985d0d2b075f6158d1ca28

loadActual: add transplantFilter method to filter what gets transplanted

view details

isaacs

commit sha 32747a258f46fbf1835fa16b381b5f2b31c4f263

reify: replace load-bundle-deps module with loadActual({root}) This makes it so that we get proper dep flags in bundled deps! Win!

view details

isaacs

commit sha 5fac0af5a7d1f821c07c610e5a81aa4d67afc9b2

buildIdeal: replace load-bundle-deps module with loadActual({root})

view details

isaacs

commit sha 213800882db2c48dcaa6f335ce61b90b512d932e

remove unused load-bundle-deps module

view details

push time in 5 days

issue commentnodejs/TSC

npm currency on Node v12

The only breaking change that is likely to be unwelcome is the automatic handling of peer dependencies, as @ljharb mentions. Otherwise, I would say that Node 14 should take in npm v7 as soon as it's GA. There are two mitigating factors to the peer deps issue that still make me think it's a good idea, though.

A new config option --legacy-peerdeps has been added, which will tell npm v7 to completely ignore peer dependencies, just like npm v6 does. (This config will not be respected in npm ls, so that users can still get alerted about invalid install trees.)

Without --legacy-peerdeps enabled, the install will fail, but not in any kind of catastrophic way, as it will fail during the ideal tree building phase, before anything is written to disk.

The first mitigating factor is that, prior to release, npm v7 will detect unresolvable tree errors, and suggest that the user try again with the --legacy-peerdeps flag. (We may decide to warn and re-attempt the resolution with this option enabled in these cases, but that strikes me as somewhat over an overstep, since the failure actually is a useful indicator of something the user needs to do.)

I think that's probably enough to make it acceptable to pull into Node 14. If others disagree, we can also ship the npm that Node.js ships with a npmrc built-in config file setting legacy-peerdeps = true by default. This isn't my preferred approach because it means that the npm bundled along with Node.js will be subtly different from the npm that a user would get by installing it directly. But if there's a huge negative reaction to the peer dep conflict detection, it is a tool in our kit.

It's somewhat disingenuous to ship npm v6 along with something considered "long term support". npm v6 is already in somewhat of a "minimum necessary maintenance" level of attention and support. There are known bugs and frequently requested features that are just too expensive to address in v6, since they would require the massive refactoring that has already gone into npm v7. I'm biased and optimistic, of course, but once npm v7 is out in the wild, it'll be hard to justify continuing to ship a package manager that is objectively worse.

mcollina

comment created time in 6 days

PR closed npm/rfcs

Npm.yml

What / Why

<!-- Describe the request in detail -->

n/a

References

<!-- Examples

  • Related to #0
  • Depends on #0
  • Blocked by #0
  • Closes #0 -->
  • n/a
+33 -0

1 comment

1 changed file

elimuhub

pr closed time in 6 days

pull request commentnpm/rfcs

Npm.yml

Hi, what is the intent of this? It doesn't look like a request for comment.

elimuhub

comment created time in 6 days

PR opened npm/arborist

add Arborist.rebuild()

Stacks on top of #81, land that first.

This refactors reify() to move bin linking and script running into the rebuild() method, and exposes it for use in 'npm rebuild'

+2552 -249

0 comment

70 changed files

pr created time in 6 days

create barnchnpm/arborist

branch : isaacs/rebuild

created branch time in 6 days

issue closedisaacs/node-glob

Section with a single option returns empty match

var glob = require("glob")
glob.sync('/*');          // returns a bunch of entries
glob.sync('{/*}'); // empty []
glob.sync('/{}*'); // empty []
glob.sync('/{*}'); // empty []

closed time in 6 days

asklar

issue commentisaacs/node-glob

Section with a single option returns empty match

Yeah, that's by design. {} without any commas is a literal {}. Try it in bash.

$ for i in {/*}; do echo $i; done
{/*}

You could argue that it's a bug in the design of the glob format, and I'd probably agree with you, I mean, it's freaking weird. But it is what it is. The goal of this module is to implement glob, not fix it.

asklar

comment created time in 6 days

push eventnpm/npm-v7-blog

isaacs

commit sha 6d43a2ea923ceb7ac2a8f8df1c38c592e55eb9ff

published arborist deep dive post

view details

push time in 7 days

push eventnpm/npm-registry-fetch

isaacs

commit sha 1bb4eb2c66ee8a0dc62558bdcff1b548e2bb9820

feat: add npm-command HTTP header

view details

isaacs

commit sha 09e540b09a951ded299ee028e7f1bd21cef5a6da

chore(release): 8.1.0

view details

push time in 7 days

created tagnpm/npm-registry-fetch

tagv8.1.0

like fetch() but for the npm registry

created time in 7 days

pull request commentnpm/rfcs

RFC: Add `npm-app-id` HTTP header

I have no objection to the basic idea, but it does need some work to get over the finish line. (Your continued engagement is encouraged and welcome, but not required, if you have other things to do :)

The next step is to discuss it in our weekly Open RFC meeting (which you're invited to attend!) to quickly discuss some of the ramifications of the change, decide next steps, and eventually fit it into our roadmap. I would like to drive towards some more clarity around implementation, alternatives, and prior art, just to have some confidence that we're making the right choices and not opening the door to supporting a mistake forever. Since this is going to be an interface between the client and server, it's important to get the spec as detailed as possible.

The implementation itself (as proposed, anyway) is relatively straightforward, and I don't expect it'll have much impact for other parts of the system, so it should be pretty easy to get landed once we nail down the spec.

Mykyta

comment created time in 7 days

issue commentnpm/npm-v7-blog

[QUESTION] How to share local/private code between apps?

A post on Workspaces is planned, and you can certainly use Workspaces for this use case, I'd imagine, though it'd be a little bit outside the target use case. Might be a bit too situation-specific for this blog series?

If you're comfortable using a monorepo, you could define your apps and all their shared deps as workspaces, so you'd have something like:

my-big-ol-mono-repo
+-- shared-deps
|   +-- foo
|   +-- bar
|   +-- baz
+-- applications
    +-- acme-customer-site
    +-- intranet
    +-- wob-blag

Then in the root package.json, have this:

{
  "workspaces": [
    "applications/*",
    "shared-deps/*"
  ]
}

That would, of course, re-publish all of your applications any time you push an update to any of them, or to any shared deps, and you'd have to configure your PaaS to use a specific folder to run each application. But it would guarantee that things stay in sync. (And there'd be nothing stopping you from also publishing some or all of those shared deps as npm packages, of course.)

mariusa

comment created time in 8 days

issue closednpm/npm-v7-blog

[QUESTION] How to share local/private code between apps?

What / Why

<!-- Describe the request in detail --> We'd like to share a few specific Vue components or .js files between multiple projects, front-end and backend (nodejs). Backend is published to a PaaS (Heroku).

There are alternatives, none good: https://www.smashingmagazine.com/2018/04/sharing-code-between-projects/ https://stackoverflow.com/questions/11278921/how-to-share-code-between-node-js-apps (don't want to go through the hassle of publishing private modules)

Bit looks good, but it requires signup to another 3rd party service. We already use github & npm.

Could npm v7 workspaces handle this? How?

Would be great to

  • have these shared files in 1 place, and when changed all projects know about them
  • committing to github (used also for deploying to Heroku) would include a copy of the files, so the project can run on Heroku too or also when others check out and they don't have the shared set of folders/files

Thanks!

closed time in 8 days

mariusa

issue commentnpm/npm-v7-blog

[QUESTION] npm install vs npm ci in NPM v7

That's a great idea! I'll add it to the list.

(Probably goes without saying, that old post from 2018 is not indicative of what'll actually land in npm v7, as it turns out.)

tl;dr - No, install will not be an alias to ci. But both commands call into Arborist.reify(), and so the delta between them has gotten a lot smaller.

felipeplets

comment created time in 8 days

issue openednpm/arborist

improvement: metadata handling could be more elegant

Right now, the ways that the Node class gets its resolved and integrity values from the Shrinkwrap and YarnLock metadata is a bit clunky and inelegant.

All of this should be handled by the Shrinkwrap class. The Node class shouldn't have to check for the existence of a meta.yarnLock property and know how to get what it needs from there.

Similarly, the saving and updating of the YarnLock class is done in Arborist.reify() right now, but should just be done automatically when needed by Shrinkwrap.save(). (Ie, if you're saving, and have a yarn lock, then update the yarn lock and save that, too.)

created time in 8 days

push eventnpm/arborist

isaacs

commit sha 8c658709c274e5ec85d0488e93d3d3ace03e2d81

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

push time in 8 days

push eventnpm/arborist

isaacs

commit sha bdb60eba2d124e78c8a23a5a8a883ff9d6fae604

fix: Handle tarball dependencies properly For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive lockfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose. This relative linking also _wasn't_ being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway. *Shrinkwrap* files *do* have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler. Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right. Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly. The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

view details

push time in 8 days

PR opened npm/arborist

fix: Handle tarball dependencies properly

Stacks on #80, land that first.

For 'file:' deps (whether directory or tarball type), Node.resolved is always absolute. It was initially set up to be relative to the node path, because it had to be stashed back to package.json. But, then we got rid of that approach in favor of a more comprehensive locckfile, and so the added complexity of constantly re-evaluating the resolved value as the node moves around serves no purpose.

This relative linking also wasn't being handled properly in tarball dependencies, resulting in attempting to fetch a tarball from the wrong location, relative to the dependent's directory in node_modules, rather than where it should be found. Since we never store Node.resolved anywhere, there's no need for it to be portable, and the portability had a bug anyway.

Shrinkwrap files do have to be portable, so the paths get relativized (and resolved) going into and out of lockfiles. But that's much simpler.

Tarball dependencies are more consistently handled, and now that the Node.resolved value is absolute and predictable, it's easier to get that right.

Tarball dependencies are properly regarded as a valid dependency resolution, provided that they come from the same target file. It's still a bit of a crapshoot when we get legacy install trees, but at least now we're not throwing out package._resolved when we load the manifest from pacote directly.

The added test case installs a tarball dependency which depends on two other tarballs: 1 in its own containing folder, and another outside of its folder. (This exact situation was a thorn in npm's side for most of 6.x, and might still be in some cases.)

+889 -133

0 comment

42 changed files

pr created time in 8 days

create barnchnpm/arborist

branch : isaacs/fix-tarball-deps

created branch time in 8 days

PR opened npm/arborist

Isaacs/rebuild bundle false

stacks on top of #78, land that first.

+455 -51

0 comment

15 changed files

pr created time in 9 days

create barnchnpm/arborist

branch : isaacs/rebuild-bundle-false

created branch time in 9 days

pull request commentnpm/cli

Better --force message, correct config.force documentation

@richardeschloss Certainly not hated by me, and likely hated a lot less than I have been for this message 😆

That is, I agree with you, but we're in the minority here, I think. It does get misinterpreted pretty often, and isn't as informative or helpful as it could be.

isaacs

comment created time in 9 days

CommitCommentEvent

push eventnpm/arborist

isaacs

commit sha 22439ce97e1005bfa2079ff36c29083a57010b31

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

push time in 12 days

pull request commentnpm/rfcs

RFC: Add `APP_ID` environment variable

npm audit provides package-lock.json in the request body, so we can't use package.json

That isn't really relevant. We modify the lockfile data that we send anyway, and have the package.json data readily at hand at that time, so it can reside in either place. I feel like "what app is this" is more appropriate for package.json to define than package-lock.json, which is a definition of the dependency package tree as it's laid out on disk.

Also, we're not likely to add a feature like this in npm v6, and we're considering a much more streamlined API for fetching bulk advisories in npm v7 anyway. So, sky's the limit :)

Mykyta

comment created time in 12 days

issue commenttachyons-css/tachyons

License should contain copyright notice

Confirmed, thanks!

isaacs

comment created time in 12 days

pull request commentnpm/cli

Fixes issue with preversion script exposing the new updated package information instead of the currently existing one

So, this patch won't be in v7, because that section of code has been rewritten, and npm-lifecycle is being put to bed for the long sleep of deprecated modules.

But here's what the preversion environment looks like in v7, which I think satisfies the need that this is addressing:

$ cat package.json
{
  "name": "name",
  "version": "1.2.4",
  "scripts": {
    "preversion": "env | grep npm_"
  }
}

$ node ../cli version patch
> name@1.2.4 preversion
> env | grep npm_
npm_config_metrics_registry=https://registry.npmjs.org/
npm_new_version=1.2.5
npm_old_version=1.2.4
npm_execpath=/Users/isaacs/dev/npm/cli/lib/npm.js
npm_config_access=public
npm_package_json=/Users/isaacs/dev/npm/foo/package.json
npm_command=version
npm_lifecycle_event=preversion
npm_config_node_gyp=/Users/isaacs/dev/npm/cli/node_modules/@npmcli/run-script/node_modules/node-gyp/bin/node-gyp.js
npm_lifecycle_script=env | grep npm_
npm_config_user_agent=npm/7.0.0-beta node/14.2.0 darwin x64
npm_node_execpath=/usr/local/bin/node
1.2.5

Let me know if there's more to do here, or if this meets your need. Thanks!

luislobo

comment created time in 12 days

pull request commentnpm/rfcs

RFC: Add `APP_ID` environment variable

Some questions.

Is APP_ID an opaque machine-generated token, or something with meaning that the user sets? If it's machine-generated, can the registry provide it, rather than the CLI?

Can we put it in package.json instead of package-lock.json, since package-lock.json is more ephemeral? (Users sometimes blow the package-lock away to get a fresh install. Maybe they'll do that less in npm v7, but who knows, habits die hard.)

Does it need to be (a) unique to each project, (b) unique to each top level project (but shared among workspaces in a monorepo, for example), (c) shared among each package in a user-defined concept of what an "app" is?

Can we spell it appid instead of APP_ID?

I think it'd be a nice simple thing for a registry to provide a npm-app-id HTTP header in the response if it doesn't get an appid, saying "this is the appid you should use", and then the CLI could just stash it in package.json if it gets one, and repeat it back on subsequent requests. A registry server would have to be smart enough to assign the same appid for each new request in a single session (since the CLI sends a bunch of requests out all in parallel without waiting for their responses), but we have the npm-session-id header, so that would be possible.

And if the user does send an npm-app-id header, then the registry would just echo it back.

This would also allow us to do other interesting things in the future, too, especially if an app could be associated with an org or user, and different types of access attached to that indicator. Since it's user-input, we couldn't trust it on its own, but in combination with a login token, we could say that a given appid associated with your organization can only be installed in your production IP block by certain user accounts, so you don't accidentally push to production when you thought you were doing a staging/local build, or something.

Mykyta

comment created time in 12 days

issue openednpm/arborist

Interesting change from npm v6: install with no shrinkwrap does a full update

In Arborist, if there's no package-lock.json, and you run reify() with no options, its default behavior is to build an ideal tree without consulting the actual tree, and without the guidance of a lockfile, meaning:

  • It'll always update anything that can be updated, including nested transitive deps.
  • It'll need to do a full tree diff.

I'm not sure if this is a feature or bug, tbh. It might be fine to say "if you want to lock down your deps, use a lockfile, it's right there in the name". However, it is a change, and one that's likely to have some ripple effects, the biggest of which will be lockfile abstainers complaining that npm install got slower.

Another approach would be to pass an option to buildIdealTree to say, if you can't load a virtualTree, then load the actualTree, and only make the changes necessary at each stage to correct problems or make user-specified changes.

If we decide this is a bug, the fix is pretty easy:

diff --git a/lib/arborist/build-ideal-tree.js b/lib/arborist/build-ideal-tree.js
index 7246e92..1f5ecd4 100644
--- a/lib/arborist/build-ideal-tree.js
+++ b/lib/arborist/build-ideal-tree.js
@@ -210,6 +210,14 @@ module.exports = cls => class IdealTreeBuilder extends Tracker(Virtual(Actual(cl
             .then(meta => Object.assign(root, {meta}))
         : this.loadVirtual({ root }))
 
+      // if we don't have a lockfile to go from, then start with the
+      // actual tree, so we only make the minimum required changes.
+      .then(async root => {
+        if (!this[_updateAll] && !tree.meta.loadedFromDisk)
+          await new this.constructor(this.options).loadActual({ root })
+        return root
+      })
+
       .then(tree => this[_mapWorkspaces](tree))
       .then(tree => {
         // null the virtual tree, because we're about to hack away at it

created time in 12 days

PR opened npm/arborist

Add support for the packageLockOnly config

This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else.

Re #46

+178 -18

0 comment

3 changed files

pr created time in 12 days

push eventnpm/arborist

Ruy Adorno

commit sha 983e7517bc427ccac9d55c912fe12444936ed830

fix: avoid throwing type errors on inventory.add Trying to load mock malformed testing data from the cli resulted in an unexpected TypeError. The issue can be reproduced by having a falsy funding property in a package.json, as seen here: https://github.com/npm/cli/blob/abdf52879fcf0e0f534ad977931f6935f5d1dce3/test/tap/fund.js#L81 This commit fixes the problem by avoiding trying to retrieve the `type` and `url` properties when having a `null` object reference. PR-URL: https://github.com/npm/arborist/pull/76 Credit: @ruyadorno Close: #76 Reviewed-by: @isaacs

view details

isaacs

commit sha c26399624cf72f20bd635973a0c1ea32907f6d10

Add support for the packageLockOnly config This tells reify() to just generate the ideal tree and write package-lock.json and package.json, but do nothing else. Re #46

view details

push time in 12 days

push eventnpm/arborist

Ruy Adorno

commit sha 983e7517bc427ccac9d55c912fe12444936ed830

fix: avoid throwing type errors on inventory.add Trying to load mock malformed testing data from the cli resulted in an unexpected TypeError. The issue can be reproduced by having a falsy funding property in a package.json, as seen here: https://github.com/npm/cli/blob/abdf52879fcf0e0f534ad977931f6935f5d1dce3/test/tap/fund.js#L81 This commit fixes the problem by avoiding trying to retrieve the `type` and `url` properties when having a `null` object reference. PR-URL: https://github.com/npm/arborist/pull/76 Credit: @ruyadorno Close: #76 Reviewed-by: @isaacs

view details

push time in 12 days

PR closed npm/arborist

Reviewers
fix: avoid throwing type erros on inventory.add

Trying to load mock malformed testing data from the cli resulted in an unexpected TypeError. The issue can be reproduced by having a falsy funding property in a package.json, as seen here: https://github.com/npm/cli/blob/abdf52879fcf0e0f534ad977931f6935f5d1dce3/test/tap/fund.js#L81

This commit fixes the problem by avoiding trying to retrieve the typeand url properties when having a null object reference.

While I considered the alternative of throwing a more descriptive error instead of having TypeError: Cannot read property 'url' of null - I think in this case not blowing up is a much more convenient and reasonable behavior.

+20 -1

1 comment

2 changed files

ruyadorno

pr closed time in 12 days

create barnchnpm/arborist

branch : isaacs/package-lock-only

created branch time in 12 days

PR opened npm/arborist

Add loadActual({ root: node })

Fix #72

This also uncovered a few lurking bugs around how we update metadata when transplanting nodes, and how dep flags (dev/optional/etc) were being set on installed bundle dependencies.

+613 -213

0 comment

14 changed files

pr created time in 12 days

create barnchnpm/arborist

branch : isaacs/load-actual-root

created branch time in 12 days

pull request commentnpm/arborist

fix: avoid throwing type erros on inventory.add

LGTM!

ruyadorno

comment created time in 13 days

issue commentnpm/arborist

[FEATURE] loadActual() should accept {root} option

Hm, doing this in the same way that loadActual does is a bit more tricky. There are a lot of cases in loadActual where we assume that node.path and node.realpath are the actual paths on the filesystem.

It'd probably be better to run the loadActual process with the root node we load, and then re-root the child/fsChild nodes once loaded. It involves a bunch of extra tree-walking to update paths, but it's a lot less bookkeeping along the way.

isaacs

comment created time in 13 days

Pull request review commentnpm/arborist

Add {complete:true} buildIdealTree option to load sw/bundles

 class Arborist extends Auditor(require('events')) {       nodeVersion: process.version,       ...options,       path: options.path || '.',+      cache: options.cache || `${homedir()}/.npm/_cacache`,

Hmm... really depends if we want that to be a "cacache default" vs an "npm default". It's kind of nice for cacache to just be agnostic about any npm-specific stuff, and ~/.npm/_cacache definitely feels npm-specific.

isaacs

comment created time in 14 days

Pull request review commentnpm/arborist

Add {complete:true} buildIdealTree option to load sw/bundles

 module.exports = cls => class IdealTreeBuilder extends Tracker(Virtual(Actual(cl     // satisfied by whatever's in that file anyway.     if (this[_depsSeen].has(node) ||         node.root !== this.idealTree ||-        node.hasShrinkwrap)+        node.hasShrinkwrap && !this[_complete])       return this[_buildDepStep]()      this[_depsSeen].add(node)     this[_currentDep] = node     process.emit('time', `idealTree:${node.location || '#root'}`) +    // if we're loading a _complete_ ideal tree, for a --package-lock-only+    // installation for example, we have to crack open the tarball and+    // look inside if it has bundle deps or shrinkwraps.  note that this is+    // not necessary during a reification, because we just update the+    // ideal tree by reading bundles/shrinkwraps in place.+    if (this[_complete] && node !== this.idealTree) {+      const Arborist = this.constructor+      const bd = node.package.bundleDependencies+      const hasBundle = bd && Array.isArray(bd) && bd.length+      const { hasShrinkwrap } = node+      if (hasBundle || hasShrinkwrap) {+        const opt = { ...this.options }+        await cacache.tmp.withTmp(this.cache, opt, async path => {+          await pacote.extract(node.resolved, path, opt)++          if (hasShrinkwrap)+            await new Arborist({ ...this.options, path })+              .loadVirtual({ root: node })

Yep! https://github.com/npm/arborist/issues/72 is an open issue to add a similar facility to loadActual(), which will make the loading of bundle deps a bit easier.

isaacs

comment created time in 14 days

pull request commentnpm/arborist

Add {complete:true} buildIdealTree option to load sw/bundles

Where did arborist landed with regards to bundledDependencies name variation? is it not supported? or is it just converted in some abstraction i might have missed?

The canonical name is bundleDependencies, no dD. This is enforced by read-package-json-fast here, and by read-package-json here. It occurs to me that we don't check for that in the manifests we get from the registry via pacote, which I'll go add now. (Since it's been enforced at publish time for many years now, it's probably fine, but still, good to be extra careful.)

isaacs

comment created time in 14 days

Pull request review commentnpm/arborist

Add {complete:true} buildIdealTree option to load sw/bundles

     "@npmcli/name-from-folder": "^1.0.1",     "@npmcli/run-script": "^1.3.1",     "bin-links": "^2.1.2",+    "cacache": "^15.0.3",

Until now, Arborist didn't use cacache directly! But cracking open tarballs requires a temp directory to extract into, and it just makes sense to use the temp directory that cacache provides, since the rest of npm does.

isaacs

comment created time in 14 days

issue closednpm/node-semver

Update `semver` dep in primary dependencies of @npm/cli

Task

Update the semver package in all primary dependencies of npm/cli

Primary Packages with Semver Dep

Completed Package Name Semver Range Current Version Next Version<br />(with semver bump)
<ul><li>[x] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> init-package-json ^1.10.3 1.10.3
<ul><li>[x] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> lock-verify ^2.1.0 2.1.0
<ul><li>[x] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> normalize-package-data ^2.5.0 2.5.0
<ul><li>[x] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> npm-install-checks ~3.0.0 3.0.0
<ul><li>[x] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> npm-package-arg ^6.1.1 6.1.1
<ul><li>[x] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> npm-pick-manifest ^3.0.2 3.0.2
<ul><li>[x] PR</li><li>[x] Landed</li><li>[x] Released</li></ul> pacote ^9.5.8 9.5.8
<ul><li>[x] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> read-installed ~4.0.3 4.0.3

Upstream Packages with Semver Dep

Completed Package Name Semver Range Current Version Next Version<br />(with semver bump)
<ul><li>[ ] PR</li><li>[ ] Landed</li><li>[ ] Released</li></ul> node-gyp ^5.0.3 5.0.3

Completed Package Updates

Primary dependencies

  • [ ] init-package-json
  • [ ] lock-verify
  • [ ] normalize-package-data
  • [ ] npm-install-checks
  • [ ] npm-package-arg
  • [ ] npm-pick-manifest
  • [x] pacote
  • [ ] read-installed

Upstream dependency will need external pull-request

  • [ ] node-gyp [upstream]

closed time in 14 days

mikemimik

issue commentnpm/node-semver

Update `semver` dep in primary dependencies of @npm/cli

Everything in the release/v7.0.0-beta branch is now using the latest semver.

mikemimik

comment created time in 14 days

issue closednpm/node-semver

[QUESTION] `semver.gt('1.0.0-rc10', '1.0.0-rc3')` returning false

Hi,

I'm using semver@5.6.0.

I'm running semver.gt('1.0.0-rc10', '1.0.0-rc3') and it's returningfalse, I'd expect the result to verify the rcXX suffix correctly but it's not.

Note, that I've ran semver.valid('1.0.0-rc10') before and these are valid semver versions.

Can anyone explain this behaviour or is it a bug?

Thank you!

closed time in 14 days

agatakrajewska

issue commentnpm/node-semver

[QUESTION] `semver.gt('1.0.0-rc10', '1.0.0-rc3')` returning false

The prerelease section is a period-separated set of identifiers, which may be either ASCII alphanumeric strings, or numeric strings. Strictly numeric strings are sorted in numeric order, and alphanumeric strings are sorted in ASCII order. (Ie, numbers, then caps alphabetically, then lowercase alphabetically).

Because rc10 and rc3 are both alphanumeric strings, and '1' is a lower ASCII code than '3', it sorts like it does.

> semver.gt('1.0.0-rc10', '1.0.0-rc3')
false
> semver.gt('1.0.0-rc.10', '1.0.0-rc.3')
true
> semver.gt('1.0.0-rc30', '1.0.0-rc3')
true
> semver.gt('1.0.0-rc30', '1.0.0-rc31')
false
> semver.gt('1.0.0-rc30', '1.0.0-rc4')
false
> semver.gt('1.0.0-rc.30', '1.0.0-rc.4')
true
agatakrajewska

comment created time in 14 days

pull request commentnpm/cli

Replace condescending line with a helpful message.

My apologies, I suppose I have been doing this so long as my normal day to day job, it's easy to forget that "land a patch in npm" can come along with some emotional weight. My intent was for that other PR to be a part of the conversation, just to have a thing to point to in discussing it, but I can certainly see how it came off as dismissive. You're not the first to suggest changing or removing this warning, but you are the one whose nudge led to that update, so I think it's fair to give you credit for it (beyond just linking to this PR in the commit). I'll update my commit to do so, if that's ok with you. I appreciate your candor in expressing how this experience landed for you emotionally, I know that's not easy to do with strangers on the internet.

By "a little judgey", I didn't intend that to be a normative/subjective value judgement. Just that it's stating a judgement, implicit in "what you are doing is not recommended". This is also factually inaccurate sometimes (for example, npm audit fix --force, which we specifically recommend in some cases!) I'm not married to the wording in #1259, and would welcome any further input or thoughts you might have on the matter.

pablocubico

comment created time in 14 days

Pull request review commentnpm/rfcs

Clarify/Outline the RFC Withdrawal Process & Amendment

 integration into the CLI may also be deferred to a later date, or a later semver-major CLI release, at the npm collaborators' discretion. All the RFC does is communicate the team's consensus to accept a change. +### Implementation+ When the changes described in an RFC have been implemented and merged into the relevant repository (and thus, due to be released), the corresponding RFC will be moved from `accepted/` to `implemented/`. If you'd like to implement an accepted RFC, please make a PR in the appropriate repo and mention the RFC in-the PR. Feel free to do this even for work-in-progress code!+the PR. Feel free to do this even for work-in-progress code.++### Withdrawal++From time to time npm collaborators will review RFCs awaiting implementation to+ensure their accuracy and relevance. In cases where a previously ratified RFC is+deemed to no longer be a viable candidate for implementation, an [**amendment+section**](withdrawn/0000-template.md) will be added to the bottom of the+document outlining the reason for repeal and subsequently moved to the

My only nit is that the amendment section should be at the top, not the bottom, so that it's harder to miss.

darcyclarke

comment created time in 14 days

push eventisaacs/minipass-pipeline

isaacs

commit sha 331ec6973f8d031724f5d831b932d5a4b40bdca7

Exert backpressure when pipeline is not flowing If the pipeline as a whole is not flowing, then it should return `false` from any write operation. Since the Pipeline listens to the tail stream's `data` event, the streams in the pipeline always are in flowing mode. However, the Pipeline itself may not be, so it would return `true` from writes inappropriately, allowing data to be buffered up in the Pipeline excessively. This would not cause any significant issues in most cases, except excess memory usage. Discovered while debugging npm/npm-registry-fetch#23

view details

isaacs

commit sha 29b2399520fa0cbe38f9932f03f9ee8205ccd12e

1.2.3

view details

push time in 14 days

created tagisaacs/minipass-pipeline

tagv1.2.3

create a pipeline of streams using Minipass

created time in 14 days

CommitCommentEvent

push eventnpm/arborist

isaacs

commit sha 1570bea52039b71d152758fac9356b5e793a4650

update pacote to pull in fixed minipass/nrf/mfh

view details

push time in 14 days

issue commentnpm/npm-registry-fetch

[BUG] cache and registry in combination makes fetch unstable

Alright! Removed the kludge in make-fetch-happen, and fixed this properly at the root.

That was a beast to track down.

pgsandstrom

comment created time in 14 days

more