profile
viewpoint

bmeck/callbackhell 9

utility for callback hell

bmeck/babel-analysis 8

SSA/CFG for babylon

bmeck/auther 5

node.js service for logins and api tokens

bmeck/atxjs-offline 3

Place for Austin Javascript Offline Application Example

bmeck/clip 3

Express meets the CLI

bmeck/--- 2

A repository that doesn't like search engines.

bmeck/candor.io 2

candor.io is a server framework that uses the candor language combined with libuv to make fast scripted network servers.

bmeck/cejs 2

cejs (pronounced Sieges) is a simple ejs compiler

issue commentnodejs/node

Disable __proto__

I know we have --frozen-intrinsics and --experimental-policies, perhaps we could make a more cohesive single set of defaults instead of a flag for each. This would be similar to the difficulty TypeScript faces with all its flags, and strict mode for TS opts into the desired but breaking defaults.

mcollina

comment created time in 2 hours

issue commenttc39/proposal-async-init

Prior Art?

In general there are a very large variety of ways that people do async initialization and there is plenty of work on prior art and opinions on what makes the differing patterns of static methods, private constructors, factories, etc. better than each alternative. I think the goal here is to find what is best about the approaches to take a step forward. I don't see anything as being able to be dismissed purely based upon lack of prior language art. For example, if async constructor is just sugar for one of those patterns I don't think it is problematic to use that as a syntax. Perhaps clarification on why async constructor is problematic as a syntax and thus not good for any form of semantics. Is the argument that we must adopt the semantics of another language's syntax being what is discussed?

rickbutton

comment created time in 3 hours

issue commentljharb/proposal-private-fields-in-in

`in` does not preclude shorthand?

#k has multiple proposals potentially using it. I was thinking about https://github.com/tc39/proposal-private-declarations as having the same usage as an environment variable as key rather than this. shortcut. Thanks for clarifying, but it seems to only preclude 1 type of forwards usage, maybe this could be called out as constraining rather than precluding usage of #foo as an expression.

bmeck

comment created time in 6 hours

issue openedljharb/proposal-private-fields-in-in

`in` does not preclude shorthand?

I'm a bit confused by:

Specifically, this is because if #brand (in the previous example) is a shorthand for this.#brand, then (#brand) in obj would have to mean the same as this.#brand in obj, which means "is the value of this.#brand in obj?", not "does obj have the private field #brand?".

Why is #brand in obj problematic? this.#brand is the reference to the property value of #brand on this so it doesn't collide with the environment reference #brand to my knowledge.

created time in 7 hours

issue commenttc39/proposal-module-attributes

Module type and content-type sniffing

Also, since the web already uses MIME to enforce format per https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-single-module-script why would we want differing behavior?

xtuc

comment created time in 7 hours

pull request commentnodejs/node

Add support for extensionless workers and ESM eval

We could use a type assertion instead, which seems to be the difference of as (which asserts the type of a target from link tag / module attributes proposal) and type (which changes the loading system of a target).

However, per things like https://github.com/nodejs/node/pull/31388 which was brought about by extension-less support; we probably want to think about how this would expand in the future for formats outside of JS itself. I think if people were uncomfortable with type: 'wasm' in package.json it likely would be argument against using a type field in the constructor here as well.

Since the ESM loader does support data:, and "bin":{"foo":"bin/foo.cjs"} within package.json can point to a file with an extension it might also just be prudent to not do anything except having a switch to select the loader used for the initial worker endpoint; this is similar to how shouldUseESMLoader works https://github.com/nodejs/node/blob/f5ef7cd22231987a73af9f02cc5f6c80b6dbbd9b/lib/internal/modules/run_main.js#L23 in a dynamic manner and wouldn't appear to cause any new forms of conflicts where resources can be interpreted in multiple different ways.

aduh95

comment created time in 7 hours

issue commenttc39/proposal-module-attributes

Module type and content-type sniffing

@xtuc Browser specs recommend never using sniffing for scriptable mimes, per https://mimesniff.spec.whatwg.org/#sniffing-a-mislabeled-binary-resource

Warning: It is critical that the rules for distinguishing if a resource is text or binary never determine the computed MIME type to be a scriptable MIME type, as this could allow a privilege escalation attack.

xtuc

comment created time in 8 hours

issue commentnodejs/modules

Node 13.9.0 blocks the use of npm scripts together with --loader funcionality

@guybedford backwards compatibility breakage came from --loader, without --loader it still worked fine

sla100

comment created time in 5 days

issue commentnodejs/modules

Node 13.9.0 blocks the use of npm scripts together with --loader funcionality

Overall, this is the intended behavior we got to after problems with having support for files lacking extensions in the ESM resolver; so I wouldn't think we need to fix this. My recommendation is you can have tsc map to a file with an extension, or expand the custom --loader supplied to support loading files without extensions (since it is overwriting how tsc works anyway).

sla100

comment created time in 5 days

issue commentnodejs/modules

Node 13.9.0 blocks the use of npm scripts

This is because --loader forces the ESM resolution algorithm not CJS resolution, and extension-less files are not resolved by default for ESM, see https://github.com/nodejs/node/pull/31415

sla100

comment created time in 5 days

pull request commentnodejs/node

module: prevent builtins from resolving from disk

Are we sure that ESM doesn't do the same currently, I can repro the node_modules lookup for fs/. Screen Shot 2020-02-20 at 11 16 56 AM

devsnek

comment created time in 5 days

issue commentnodejs/modules

import.meta.main

@ljharb I'd agree that question isn't solved here; if there is a more universal solution we might want to look at other things. I think adding conditions for things like --require style usage would be nice but likely is a different topic and might be better served with a different mechanism than how an entry point is bootstrapped. Even if we call main() it doesn't mean that the file is the only entrypoint, and userland could always call main() on other modules manually.

MylesBorins

comment created time in 6 days

issue commentnodejs/modules

import.meta.main

We might also want to lock step with WASM interface-types so that WASM side doesn't go out of sync with ESM

MylesBorins

comment created time in 6 days

issue commentnodejs/modules

import.meta.main

@devsnek for WASI, it is just the "main" string.

MylesBorins

comment created time in 6 days

issue commentnodejs/modules

import.meta.main

WASI is going to be taking the main() route (different from start); this might be something to discuss.

MylesBorins

comment created time in 6 days

issue commentAgoric/agoric-sdk

use Node's ES-module support, stop using `esm`

Unless there is urgent reason to move to native ESM, I'd stick to compiling until Node's ESM is out of experimental status. Testing/utility frameworks have trouble with real ESM in particular if you do things like: stubbing, mocking, hot module reloading, cache invalidation, etc. There is not real path currently towards fixing that at the spec level so the only alternative is to use a runtime loader for those workflows which is back to a compilation step and Node's loader APIs are not stable.

warner

comment created time in 7 days

issue commentrbuckton/proposal-refs

Exclusive Refs?

@rbuckton I was specifically thinking about http://www.cplusplus.com/reference/memory/shared_ptr/ . Being able to ensure async control flow in JS does not interleave in odd ways would be nice:

if (foo === 1) {
  await bar();
  // foo is not guaranteed to be 1 here currently
}
bmeck

comment created time in 8 days

issue openedrbuckton/proposal-refs

Exclusive Refs?

With this proposal would it be possible to create an exclusive reference where you can guarantee a binding/Reference has exclusive access to a variable. I'm more just poking at the ideas of smart refs and trying to see how they would work with this proposal.

created time in 8 days

issue commenttc39/proposal-async-init

Suggested problem solution that avoids awaiting in constructors

@brad4d

I'm not sure what you mean when you say "ergonomic integration". Can you elaborate?

A lot of JS language progression is in how usable certain workflows are and that is the core of lots of features being investigated such as this, previous features such as await, etc. The core here is that various features of JS do not work well with asynchronous initialization.

Constructors expect to receive actual data to be stored. They are only responsible for storing it correctly in the object being constructed.

This is not how JS works today so it doesn't seem really relevant as we cannot change that fact. JS has constructors that perform logic and can contain side effects. The Promise constructor is an example of JS builtin that is explicitly for that purpose of dealing with something with side effects and creating them when the Promise is initialized.

There is truth that this proposal does seek to address issues with workflows people use that do perform logic during construction, but JS the language does explicitly allow logic during construction and that cannot be changed. We should seek to understand the problems people have and serving them.

Authors of JS books even go into detail on how to perform logic and behaviors during initialization so it isn't a strange idea to perform logic, side effects, or async initialization. See https://2ality.com/2019/11/creating-class-instances.html for an example of a prevalent author doing so.

brad4d

comment created time in 8 days

issue commenttc39-transfer/proposal-async-init

Suggested problem solution that avoids awaiting in constructors

I believe the goal of this proposal is to be able to write code that looks something like this [...]

At stage 1 the goal of this proposal is to find out what problems there are with asynchronously initializing instances of fields. It does not tailor to a specific design direction until stage 2. The problems come from class features like class fields/accessors/etc. not having ergonomic integration with asynchronous creation of classes.

Does the code pattern below solve the problem this proposal addresses?

That uses an inversion of control to avoid asynchronous initialization entirely by relying on consumers to perform all asynchronous initialization. This investigation is partly about how to allow code to be written such that asynchronous resources do not require inversion of control. There are a variety of potential workflows we could pursue even if we do end up requiring such an inversion of control. The standardization of asynchronous initialization could be of any reasonable design and if you want to argue for inversion of control, perhaps explaining how to perform ergonomic integration with class fields and accessors such that the class author can ensure encapsulation and allow for asynchronous defaults etc. would help.

brad4d

comment created time in 12 days

issue commenttc39-transfer/proposal-async-init

await super() syntax

super await() is invalid currently so it could be used.

Also, despite the fact that await super() is a valid syntax, it will work exactly as expected when applied to an async superclass constructor call in an async subclass constructor (except the case when superclass is not async, but has Promise-like instance).

Unfortunately this is likely not true, the this value of the constructor remains the Promise instance instead of the result of awaiting the Promise. That means that things like class fields are attached to the Promise instead of the asynchronous value they are wrapping.

miyaokamarina

comment created time in 12 days

issue commentnodejs/modules

Expose loader class to userland

@GeoffreyBooth Likely they are separate issues and people creating their own Loader instance (not --loader integration) would need to call out to a different resolution behavior as a fallback somehow normally.

jkrems

comment created time in 12 days

issue commentnodejs/modules

Expose loader class to userland

I'd want to audit to check that users can't mutate the default loader (including proto pollution), but exposing the type it seems fine to me as long as you don't instrument the active realm's loader. Would the idea here be to automatically set the "parent" to the current realm's loader instead of the default node loader? Will need to think a bit on that.

jkrems

comment created time in 12 days

issue commenttc39-transfer/proposal-async-init

Naming

@Dimtime at stage 1 I have no strong feelings on the eventual design. However, I do believe since static fields/methods/accessors are available without any sort of guard they will likely be prohibited to use await. Unlike instance properties, static properties wishing to ensure they are available and don't have an incomplete class can use await when creating the class itself.

const _foo = await connect();
async class X {
  static foo = _foo;
}

I'm open to alternative opinions, but it seems the ability to form a complete class derived from async operations is simpler than an instance. The problems with instances largely come from encapsulation and inheritance that do not affect construction of the class itself. If reasonable semantics for await in classes is proposed it can be done as a follow on most likely.

Dimtime

comment created time in 12 days

issue commenttc39-transfer/proposal-async-init

Naming

@Dimtime this proposal might cover things outside the constructor like fields. I'm not sure that argument is really relevant if we continue to find things outside the constructor that would be good to support. E.G.

class X {
  foo = await connect();
}
Dimtime

comment created time in 12 days

issue commentbmeck/proposal-async-init

Surely more justification needed, against existing alternative patterns!?

Unfortunately a variety of class features are starting to struggle when integrating against other workflows. This proposal is to seek what problems exist with asynchronous workflows and streamline them such that they are easier and more standardized workflows.

I am open to any approach to make asynchronous initialization better, but this issue does not seem to state any direction to take:

inconsistency between async vs sync classes can lead to mistaken usage in either case!

This needs to be backed up on how this is the case so that an argument for/against the premise can be made. Of note, it seems to have a parallel in async vs non-async functions and clarity in how the problem exists for classes but not for functions would be helpful.

On the issue of alternative patterns that already exist, I like to do this

A variety of workflows do exist for async initialization! However, in particular things like private fields are problematic to work with:

class Query {
  #ready;
  results;
 constructor() {
    async function init() {
      // setup #ready and await it
    }
    return init();
  }
}
class SpecializedQuery {
  special; // installed on the Promise<Query> not on Query
  constructor() {
    // cannot access #ready
  }
  get firstResult() {
    // cannot await here, so must return a Promise
  }
}

The proposal is not concerned with the many existing ways to achieve some result; this proposal, like you state, is about the ease of learning/preventing bugs and any missing capabilities. The more complex and/or intricate the workaround to make JS features work together, the more value this proposal seeks to give by standardizing that workflow (whichever it may be).

If the concern is about producing new error prone workflows, I'd happily discuss that; e.g. if it could be shown that the proposal (or one of the possible designs in the FAQ) does new behavior that is problematic or error prone concretely. Doing so would allow discussion and narrowing of what the proposal could need to do to avoid the issue.

TheNavigateur

comment created time in 13 days

issue commentbmeck/proposal-async-init

Naming

Can you clarify how this differs from async vs non-async functions? e.g. async functions return promises, but do not use the word promise

Dimtime

comment created time in 13 days

create barnchbmeck/proposal-async-init

branch : master

created branch time in 14 days

created repositorybmeck/proposal-async-init

created time in 19 days

pull request commenttc39-transfer/proposal-csprng

Add spec text to be compatible with WebCrypto

I have no opinion on breakage, just want to keep it recorded as being thought about.

rbuckton

comment created time in 19 days

pull request commenttc39-transfer/proposal-csprng

Add spec text to be compatible with WebCrypto

be aware that the default node repl exposes builtin modules as globals and crypto is the name of one of them.

$ echo 'typeof globalThis.crypto' | node -i
Welcome to Node.js v12.13.1.
Type ".help" for more information.
> typeof globalThis.crypto
'object'
rbuckton

comment created time in 19 days

issue commentnodejs/node

Hashbang not resolving `--loader` relative to entry

Interpreter directives (Hashbangs / Shebangs) do not parse consistently across shells. I do not believe we should attempt to specify behavior within node itself due to this as node's behavior could differ from shell behavior.

SMotaal

comment created time in 22 days

issue commenttc39/proposal-module-attributes

Define what a type is

One thing that is valuable is for personal/vendor specific types to be clearly defined. Using shortnames like mime groupings from WHATWG would give us a central registry to coordinate across environments, but doesn't really specify how to specify non-standard extensions. MIME types themselves have prs., vnd., and x. for non-standard types as well as a means to register new types on a standards track. I think it may require some more typing but there is certainly more flexibility for non-standard usages as well as a clear means for non-web usage to expand a standard set of known types if we use MIME.

xtuc

comment created time in 22 days

issue commentnodejs/modules

Why aren’t built‑in ESM exports synced automatically?

This was the case originally, but problem with accessors (particularly getters) caused issues. See:

  • https://github.com/nodejs/node/pull/29426
  • https://github.com/nodejs/modules/blob/master/doc/meetings/2019-09-25.md#nodejsnode
  • https://github.com/nodejs/node/pull/29737
ExE-Boss

comment created time in 25 days

issue commentnodejs/security-wg

Introducing Audit Hooks to Node.js

@vdeturckheim that seems reasonable, but we would want to make sure we document all the bindings we expose and I'd still be concerned with global mutations. I know we have primordials now but that seems extremely inconvenient/error prone to have developers code against, I know we are working on moving ESM Loaders to a separate thread for isolation. IDK how others feel if enabling this feature had isolation even if just in a separate Isolate since that would affect performance and we could not pass non-primitives across isolates without cloning.

vdeturckheim

comment created time in a month

issue commentnodejs/security-wg

Introducing Audit Hooks to Node.js

@vdeturckheim I guess I should clarify my concerns. My concern is mutable globals/builtins affecting audit hooks. E.G. Someone replacing globalThis.Promise and/or something like fs.readFile to intercept, prevent, or otherwise manipulate the audit hook and prevent its intended purpose when such a hook uses those APIs (such as reading a config file off disk and/or using Promise.all, etc.).

If the mutation is able to remove the hook without directly manipulating the actual hook code, that is problematic I would think. The only way that seems safe is if you audit all your hook code, allow it to run prior to any untrusted code, and have a way to ensure they have access to the primordials they need (I wouldn't want to encourage process.binding as a way to create a robust fs.readFile).

I'm not really worried about performance but about the reliability of the feature itself. If it isn't reliable, we probably shouldn't encourage it.

vdeturckheim

comment created time in a month

issue closedhax/proposal-function-this

Clarification of inner closures

Inner closures like the direct eval example in the README seem to need some extra clarification. Particularly I would expect:

function foo() {
  const bar = () => this;
}

foo.thisArgumentExpected to equal true since the this comes from the foo environment record specifying that it does have a this binding per HasThisBinding and bar resolving a reference to that statically.

closed time in a month

bmeck

issue commenthax/proposal-function-this

Clarification of inner closures

Yes, thanks

bmeck

comment created time in a month

issue commentnodejs/security-wg

Introducing Audit Hooks to Node.js

This seems reasonable and aligns with the vision I have had. That said, I do think any such API needs to be much more robust than what we often do in Core these days with mutable internals.

vdeturckheim

comment created time in a month

issue commentnodejs/modules

Support for Import Maps

What could we do to get the equivalent functionality while avoiding this issue?

I do not have an answer to that question due to the above behavior of overtaking anything we might propose as separate.

wesleytodd

comment created time in a month

issue commentnodejs/modules

Support for Import Maps

More interest from whom?

People wanting to implement import maps in other environments <-> the import maps proponents. As of yet, we have some interactions about how other environments may use what the web wishes to use, but not about what the web can allow to let other environments have their own use cases.

I was hoping to make that case, are there specific things you think I could expand upon above which would better make the case for you? I have mentioned that I have a PoC of one key feature it has that current solutions do not provide. If I provided some more use case examples than my very simple gist would that help?

One of the big things is composition being discussed at length. E.G. Placing something under node_modules/ means that the location in which a program is being run affects its import map. Since multiple node_modules/ folders may exist (or none) this means discussion of how multiple import maps needs to take place and likely cannot be put off like in the browser.

Other data such as integrity strings would also be of benefit to discuss. I don't think the path resolution is the only thing that will be used when doing imports. In the future, browsers are also looking at requiring module attributes and those likely also need to be discussed.

I think providing use cases that elaborate on comparisons with existing Node features and explanation of why import maps are a better fit than a Node specific alternative would be helpful for these kinds of things.

People matter, and if you don't have faith in their process is there something we could do to move the specification into a better home?

This is part of the issue, things like RFCs have sometimes been taken over and superseded by WHATWG claiming to be the source of truth rather than the original RFC. I doubt moving the specification would prevent such behavior in the future.

wesleytodd

comment created time in a month

issue commentnodejs/modules

Support for Import Maps

That said, Node should absolutely support any standards

I'd be against stating just because something is a standard it is good to land it everywhere. The implications of import maps and the integrations across environments does not seem to be well discussed. If we saw some interest in import maps beyond the current browser heavy path forward to address things like those in this issue such as having more data, composition on a package level, etc. I'd be more interested in that specific feature. As it stands things, Node is not beholden to follow any given standard it does not feel is a good fit for the runtime; and, as it stands I do not see import maps as being something that is well suited towards being merged. Though, I would state I am generally biased against WHATWG controlled features in general these days and more so biased against them the more interactions I've had over the years so it isn't like I'm a neutral party here.

wesleytodd

comment created time in a month

issue commentnodejs/modules

Support for Import Maps

@wesleytodd see stuff like https://github.com/nodejs/node/pull/29527 which would add a benchmark to node specifically about that feature.

wesleytodd

comment created time in a month

issue commentnodejs/modules

Support for Import Maps

So, we already have this feature of static resolution mappings to some extent using https://nodejs.org/api/policy.html#policy_dependency_redirection ; in all honesty, even though I wrote the feature and have spent some time tuning performance, the benchmarks there are not pleasant when you run them on medium sized apps particularly regarding startup time. I think we should be more critical about the benefits here as it is not a sure win in terms of performance to merely have static resolution.

If we could also address the other concerns about the limitations of only having data for resolution and adding additional meta-data that seems fine, but starts to seem to just be working towards a different policy file.

wesleytodd

comment created time in a month

issue openedhax/proposal-function-this

Affect of inner closures

Inner closures like the direct eval example in the README seem to need some extra clarification. Particularly I would expect:

function foo() {
  const bar = () => this;
}

foo.thisArgumentExpected to equal true since the this comes from the foo environment record specifying that it does have a this binding per HasThisBinding and bar resolving a reference to that statically.

created time in a month

issue commenttc39/proposal-module-attributes

Use general key-value syntax, or a single string?

I've stated in other issues that only tailoring to the web specific type attribute is not appealing. If the intent is not to include the ability to have multiple attributes in this proposal but instead in a follow on proposal I would not be comfortable moving forward.

littledan

comment created time in a month

issue commentnodejs/node

Feature: Immutable Buffer `buffer.readonly()`

It might be more prudent to first go through TC53 instead of TC39]first which likely has a bigger interest in things like this for putting values into ROM as much as possible unlike most desktop runtimes. That might at least be a first step to gather implementer interest before presenting TC39 a proposal.

mikeal

comment created time in a month

issue commenttc39/proposal-record-tuple

Concerns of value types using prototypes.

@littledan we saw presentations about issues with integer indexed types at TC39 and issues of how they are a great source of issues. I'm not necessarily convinced in light of that presentation that duplicating some pattern that has historically caused issues is a good path to take.

bmeck

comment created time in a month

issue commenttc39/proposal-record-tuple

Holes in Tuples

@littledan I do not believe my comment was off topic as it does have a position on how this specific issue could be avoided. Discussing if this issue is something that needs to be addressed is intertwined with other issues and removing the ability to cross reference issues seems a bit much.

caitp

comment created time in a month

PR closed nodejs/node

Reviewers
esm: add package type wasm C++ ES Modules modules-agenda

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

  • [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
  • [x] tests and/or benchmarks are included
  • [ ] documentation is changed or added
  • [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

The main benefit of this is to allow files without an extension to be treated as WASM. This does alter the logic for detecting if something should be treated as going through the ESM loader for the main module slightly by inverting the condition from checking for "module" to instead check it doesn't match "commonjs"

This remains flagged behind the WASM modules flag

+81 -32

25 comments

6 changed files

bmeck

pr closed time in a month

pull request commentnodejs/node

esm: add package type wasm

https://github.com/nodejs/node/pull/31415 is removing support for any extension-less files and supersedes this for now. A different PR can be opened if other features are found to warrant this feature.

bmeck

comment created time in a month

issue commentnodejs/modules

Docs/review of various affects of type field

That sounds like a bug or at least unexpected behavior. The loader for the entry point should be determined by the file format of the entry point, not by the closest package.json#type. E.g. if I'm in type:module and run node foo.cjs, I wouldn't expect the import loader to be involved.

It is if a .cjs or .mjs is seen, but the algorithm is actually a bit more complex to reason about than just that, see shouldUseESMLoader's impl

bmeck

comment created time in a month

issue openednodejs/modules

Docs/review of various affects of type field

"type" in package.json has a few affects:

  • when loading the main entrypoint of the process or a worker thread it can determine which loader to use: ESM loader (if package.type is "module") vs CJS loader (if type is anything else? [this seems a forwards compat bug?] )
  • manipulates format resolution within its package boundary
    • .js and extension-less files in particular
    • also .node
  • causes the CJS loader to error for CJS format files that end in .js within that package boundary (not extension-less files?)

We should probably document and/or bugfix these if they are not the expected behavior.

created time in a month

push eventbmeck/node

Bradley Farias

commit sha 6acc1f517d75d24c2665a08264443eba76898d88

make sure package type fallback works the same for CJS loader

view details

push time in a month

issue commentnodejs/modules

Rewrite modules readme

@weswigham I'd gladly cede the proposal over to you, as I do not share the confidence of it moving forward due to it affecting SourceTextModuleRecord being a requirement to my knowledge unless node ships only DynamicModuleRecords and doesn't use SourceTextModuleRecord. Let me know if you want to do such as I'm not going to queue it on the agenda anytime.

GeoffreyBooth

comment created time in a month

issue commentnodejs/modules

Rewrite modules readme

@weswigham to my knowledge TC39 wanted a different approach from the Dynamic Modules proposal which would affect existing module graphs so idk where/what we could invest in currently. I'm listed as the champion but was pondering closing that proposal as I don't see a path forward for that one at least.

GeoffreyBooth

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

It sounds like .wasm is equivalent to .mjs as far as end users are concerned (both are imported via the script type=module, import, etc.).

I don't understand this. Not for/against any part of the comment; I simply am confused.

jkrems

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

The field in package.json determines the Content-Type of ambiguous files.

This is not purely the case, it does have potential alterations to other things like which loader to use for the main entry point even if the file of the main entry point is not "ambiguous".

jkrems

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

So it sounds like browsers are treated WASM the same way they treat ES module JavaScript.

No, they remain decoupled in terms of format. They use the same loader workflow though.

Is there an issue with Node treating .wasm files the same way we treat .mjs files?

Yes, that would not allow WASM to be loaded. I don't think I understand this question fully.

In that they can only be used via import/import()

This is not true today, per the WebAssembly global.

and they are always parsed the same way regardless of "type".

This likely won't be possible if WASI/WASM fully diverge.

jkrems

comment created time in a month

pull request commentnodejs/node

esm: add package type wasm

I would also like to point out that WASM vs JS is seen as a security concern by some and is part of the discussion of https://github.com/tc39/proposal-module-attributes , to people of that mindset it might be undesirable to have any JS in their application.

bmeck

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

How will browsers support loading WASM? Via import statements? Will they also support WASM via <script> tags, and if so, what would the type of such a tag be?

Yes, using script tags (type=module) and/or import. The type wouldn't determine the format/parser used, it determines the loader (classic vs module , though other kinds exist).

I assume WASM has only one parse goal?

Yes, but it is starting to have 2 distinct semantic goals (WASI vs WASM).

jkrems

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

I thought that all WASM files were in parse goal Module, are they not?

They are not. They are a completely different parser.

jkrems

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

@jkrems at least in my last call with Dan they wanted WASI and WASM to be in the same file extension.

jkrems

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

@GeoffreyBooth

How would people categorize all of our other natively supported types: .node, .json, .wasm? ...

I'd like to be wary of attempting to see if something "is a Module" (unclear if meaning an Abstract Module Record here or in "type":"module"). WASM can be loaded through import but also just via an API. Virtually anything can be transformed into an Abstract Module Record and many things can be exposed without using Module Records at all (JSON/WASM/etc.).

I don't fully understand the comment.

jkrems

comment created time in a month

issue commentnodejs/modules

The type field and `.node`

If memory serves it was pulled out due to compat concerns. Don't remember when, but seems we should discuss what type "module" is intended for rather than adding .node to it prematurely

jkrems

comment created time in a month

push eventbmeck/node

Bradley Farias

commit sha 9fc929c6010f48108ef7cd6800bfafe341801bf1

remove .js resolution from wasm type

view details

push time in a month

pull request commentnodejs/node

esm: add package type wasm

That means we can effectively still say it's the field that determines the type for all ambiguous files since few people will try to put WASM into .js in practice.

I do not want the focus of the PR to be on preserving this idea. Already this is not true (see .node in "commonjs") and other features toggle in the presence of type like if to use the ESM loader for the main module. If that is a invariant we wish to make we seem to have already failed and would probably need the loader to be reverted to have that invariant in other PRs.

My desire is for the focus of this PR is solely on ensuring we have a way for WASM to be supported at the same level as other first class sources by enabling loading WASM from files without extensions.

bmeck

comment created time in a month

pull request commentnodejs/node

esm: add package type wasm

We could also just not give .js a format, and then it would error. .mjs and .cjs would still work. Seems better than forcing a new extension onto WASM and we could then ask the MIME owner about adding .js if truly desired for this pr to land.

On Thu, Jan 16, 2020, 5:19 PM Jan Olaf Krems notifications@github.com wrote:

I'm not opposed to support for running a purely WASM application. But the field name is awkward to me. Right now it can be read as "(content) type of inherently ambiguous files like .js and files lacking an extension". With this change, it's "(content) type of extension-less files and sometimes also of .js, you have to know from context if that applies".

I think I would even prefer if this PR would treat .js within those packages as WASM. That would be consistent with what the field currently does. That wouldn't break people who want to ship a purely WASM app and everyone else can use .mjs.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nodejs/node/pull/31388?email_source=notifications&email_token=AABZJI4BQSR4RVOO3A7K7YLQ6DTRFA5CNFSM4KH2TNG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJF4YXY#issuecomment-575392863, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABZJI3LS4NSYQV6A2YMEMLQ6DTRFANCNFSM4KH2TNGQ .

bmeck

comment created time in a month

pull request commentnodejs/node

esm: add package type wasm

@jkrems how is ".js" ambiguous? It has a specified value (which is "module" in this case as you guessed)

But that opens a whole can of worms about configuring all kinds of file extension mappings. I'm pretty close to saying "do we need to support extension-less wasm"?

The same could have been said about extension-less ESM. I'm not sure how wanting to run a purely WASM application would be considered a bad thing given WASI and the like coming in the future.

bmeck

comment created time in a month

pull request commentnodejs/node

esm: add package type wasm

@devsnek I think that passing things around to the resolver would be a different API/PR. People have already asked about a getPackageType function in other issues and that might be simpler.

bmeck

comment created time in a month

pull request commentnodejs/node

esm: add package type wasm

@devsnek package types are format mappings, this just adds another set of format mappings to allow files without extensions to be seen as WASM.

bmeck

comment created time in a month

PR opened nodejs/node

esm: add package type wasm

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

  • [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
  • [x] tests and/or benchmarks are included
  • [ ] documentation is changed or added
  • [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

The main benefit of this is to allow files without an extension to be treated as WASM. This does alter the logic for detecting if something should be treated as going through the ESM loader for the main module slightly by inverting the condition from checking for "module" to instead check it doesn't match "commonjs"

+81 -31

0 comment

5 changed files

pr created time in a month

push eventbmeck/node

Bradley Farias

commit sha 2e2db34530f02d6dda9db14756e21608a45d5e8b

esm: add package type wasm

view details

push time in a month

create barnchbmeck/node

branch : package-type-wasm

created branch time in a month

pull request commentnodejs/node

WIP: Move ESM loaders to worker thread

@jkrems I'm somewhat in agreement as long as we can agree that using both is a stopgap that needs to be filled eventually

bmeck

comment created time in a month

pull request commentnodejs/node

WIP: Move ESM loaders to worker thread

@coreyfarrell a separate PR would be doable, but it likely should get a docs deprecation as we don't have a full replacement for some behavior of attenuating globals in the same way.

bmeck

comment created time in a month

issue commentnodejs/modules

import(cjs) messes with module.parent

@SimenB not all ESM have a valid file path associated with them (data: in particular for now), and/or there may have multiple ESM with the same file path but different URLs (using URL fragments). I'm not sure we could preserve that through populating module.parent from an ESM base.

aduh95

comment created time in a month

issue commenttc39/proposal-record-tuple

Why restrict immutable types to only containing other immutable types?

@devsnek I presume if the optimization is applied to mutable forms such as existing types it would still require backrefs. All of this requires very carful GC optimization and I don't find talking about the potential for complex workarounds to be compelling vs a static guarantee and reliable perf. I'm not entirely sure how IOT engines etc would feel about this as well since they do not have the complex JIT VMs that are on larger devices. Reliable perf is one of the weakest parts of JS and being able to find some leeway to work towards it seems extremely valuable.

Jamesernator

comment created time in 2 months

issue commenttc39/proposal-record-tuple

Why restrict immutable types to only containing other immutable types?

@devsnek the same flag could apply to existing type, but we don't see that to my knowledge in reality? it might not be practical to do implement such a flag either as it requires all backrefs in the heap to propagate the flag

Jamesernator

comment created time in 2 months

issue commenttc39/proposal-record-tuple

Why restrict immutable types to only containing other immutable types?

If we avoid allowing non-primitive/value type references inside records and tuples it seems a GC wouldn't need to traverse them. I think this could be a very valuable property and might play into this discussion more as that would alleviate some need for allocation pools and allow very large numbers of these types without affecting GC.

Jamesernator

comment created time in 2 months

issue commenttc39/proposal-record-tuple

Holes in Tuples

I know we have had some people ask about if the prototype should be null for these types. If it is, this doesn't seem to be an issue?

caitp

comment created time in 2 months

issue commentnodejs/node

Worker threads: ability for tooling to pass data to workers and inject NODE_OPTIONS

I assume loader hooks thread will always start before any --require's are preloaded and those preloads would not be run inside the loader worker thread? Sorry if this is answered in the PR I really haven't had a chance to dig into the code yet.

That is my presumption yes.

coreyfarrell

comment created time in 2 months

Pull request review commenttc39/ecma262

Normative: Make super() throw after evaluating args

 <h1>Runtime Semantics: Evaluation</h1>         <emu-alg>           1. Let _newTarget_ be GetNewTarget().           1. Assert: Type(_newTarget_) is Object.-          1. Let _func_ be ? GetSuperConstructor().+          1. Let _func_ be ! GetSuperConstructor().           1. Let _argList_ be ? ArgumentListEvaluation of |Arguments|.+          1. If IsConstructor(_func_) is *false*, throw a *TypeError* exception.

This seems ok. I was concerned with revocation of super constructor proxies inside arguments but don't see any issues with them upon closer inspection.

bmeck

comment created time in 2 months

Pull request review commenttc39/ecma262

Normative: Make super() throw after evaluating args

 <h1>Runtime Semantics: Evaluation</h1>         <emu-alg>           1. Let _newTarget_ be GetNewTarget().           1. Assert: Type(_newTarget_) is Object.-          1. Let _func_ be ? GetSuperConstructor().+          1. Let _func_ be ! GetSuperConstructor().           1. Let _argList_ be ? ArgumentListEvaluation of |Arguments|.+          1. If IsConstructor(_func_) is *false*, throw a *TypeError* exception.

Doesn't this PR remove GetSuperConstructor from ever returning an abrupt completion?

bmeck

comment created time in 2 months

issue closednodejs/node

Workers don't start up if main thread waits

<!-- Thank you for reporting a possible bug in Node.js.

Please fill in as much of the template below as you can.

Version: output of node -v Platform: output of uname -a (UNIX), or version and 32 or 64-bit (Windows) Subsystem: if known, please specify the affected core module name

If possible, please provide code that demonstrates the problem, keeping it as simple and free of external dependencies as you can. -->

  • Version: 12.13.1
  • Platform: osx
  • Subsystem: worker_threads

<!-- Please provide more details below this comment. -->

Worker spins the event loop while starting up to pipe some events around. This means you you spin up a worker and use a SharedArrayBuffer to wait it deadlocks if the events don't finish, even if you do spin the loop a little. e.g. :

const { Worker } = require('worker_threads');
const sab = new SharedArrayBuffer(4);
const int32 = new Int32Array(sab);
new Worker(`(${() => {
  new Int32Array(
    require('worker_threads').workerData.sab
  )[0] = 1;
}})()`, {
  eval: true,
  workerData: {sab}
});
// can set this to a longer time (100ms) to get a guaranteed worker spawn on my machine
setTimeout(() => {
  Atomics.wait(int32, 0, 0);
  console.log(int32[0]);
}, 0);

closed time in 2 months

bmeck

issue commentnodejs/node

Workers don't start up if main thread waits

ah, yes.

bmeck

comment created time in 2 months

issue openednodejs/node

Workers don't start up if main thread waits

<!-- Thank you for reporting a possible bug in Node.js.

Please fill in as much of the template below as you can.

Version: output of node -v Platform: output of uname -a (UNIX), or version and 32 or 64-bit (Windows) Subsystem: if known, please specify the affected core module name

If possible, please provide code that demonstrates the problem, keeping it as simple and free of external dependencies as you can. -->

  • Version: 12.13.1
  • Platform: osx
  • Subsystem: worker_threads

<!-- Please provide more details below this comment. -->

Worker spins the event loop while starting up to pipe some events around. This means you you spin up a worker and use a SharedArrayBuffer to wait it deadlocks if the events don't finish, even if you do spin the loop a little. e.g. :

const { Worker } = require('worker_threads');
const sab = new SharedArrayBuffer(4);
const int32 = new Int32Array(sab);
new Worker(`(${() => {
  new Int32Array(
    require('worker_threads').workerData.sab
  )[0] = 1;
}})()`, {
  eval: true,
  workerData: {sab}
});
// can set this to a longer time (100ms) to get a guaranteed worker spawn on my machine
setTimeout(() => {
  Atomics.wait(int32, 0, 0);
  console.log(int32[0]);
}, 0);

created time in 2 months

Pull request review commentnodejs/node

WIP: Move ESM loaders to worker thread

 parentPort.close(); publicWorker.parentPort = null; publicWorker.workerData = null; const esmLoader = require('internal/process/esm_loader');-const {defaultGetFormat} = require('internal/modules/esm/get_format');+const { defaultResolve } = require('internal/modules/esm/resolve');+const { defaultGetFormat } = require('internal/modules/esm/get_format');++const defaultLoaderWorker = {

this probably shouldn't be called a "worker"

bmeck

comment created time in 2 months

pull request commentnodejs/node

WIP: Move ESM loaders to worker thread

I think there are a lot of missing hooks right now we can discuss in other PRs that could even block this PR. This PR however seems complex enough already without adding more hooks to it. Does that seem fine to split up that work @guybedford ?

bmeck

comment created time in 2 months

push eventbmeck/node

Bradley Farias

commit sha 0180425bb1ac2366c0a65f9fca5f01af0ec97391

remove debugging

view details

push time in 2 months

push eventbmeck/node

Bradley Farias

commit sha a21a22547a3c2eedb77f5082a522d394eb32bf2a

accidental rebase of domain

view details

push time in 2 months

Pull request review commentnodejs/node

WIP: Move ESM loaders to worker thread

 const { WeakReference } = internalBinding('util');  // Overwrite process.domain with a getter/setter that will allow for more // effective optimizations-const _domain = [null];+// Never directly access _domain, use the getter to ensure a consistent hook+// for user code+let _domain = null;

bug from a rebase on another branch ive been dealing with

bmeck

comment created time in 2 months

Pull request review commenttc39/test262

super evaluation order tests

+// Copyright (C) 2019 Jordan Harband. All rights reserved.+// This code is governed by the BSD license found in the LICENSE file.+/*---+esid: prod-SuperCall+description: SuperCall should evaluate Arguments prior to checking IsConstructable+info: |+  GetSuperConstructor ( )++  [...]+  1. Let _func_ be ! GetSuperConstructor().+  2. Let _argList_ be ? ArgumentListEvaluation of |Arguments|.+  3. If IsConstructor(_func_) is *false*, throw a *TypeError* exception.+  [...]+features: [default-parameters]+---*/++var i;+class A extends {} {+  constructor() {+    super(i = 123);

looking at your snippet you might be right. changing to null should work

ljharb

comment created time in 2 months

push eventbmeck/node

Bradley Farias

commit sha 442cb8bbce4f8674d8eaa21c2f877bf1f4905c70

wip, starting cleanup for draft PR

view details

Bradley Farias

commit sha 5a0574596f3ff7efe0d3dcd03f2fa452bdeaf5dc

remove accidents

view details

push time in 2 months

Pull request review commenttc39/test262

super evaluation order tests

+// Copyright (C) 2019 Jordan Harband. All rights reserved.+// This code is governed by the BSD license found in the LICENSE file.+/*---+esid: prod-SuperCall+description: SuperCall should evaluate Arguments prior to checking IsConstructable+info: |+  GetSuperConstructor ( )++  [...]+  1. Let _func_ be ! GetSuperConstructor().+  2. Let _argList_ be ? ArgumentListEvaluation of |Arguments|.+  3. If IsConstructor(_func_) is *false*, throw a *TypeError* exception.+  [...]+features: [default-parameters]+---*/++var i;+class A extends {} {+  constructor() {+    super(i = 123);

@leobalter in current spec this test fails, it is for a normative PR from top comment

ljharb

comment created time in 2 months

issue commenttc39/proposal-intl-displaynames

Stage 3 review

LGTM spec wise, there are a couple of README PRs outstanding it looks like though; nothing to block on

littledan

comment created time in 2 months

PR opened nodejs/node

WIP: Move ESM loaders to worker thread ES Modules modules-agenda work in progress (WIP)

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

  • [ ] make -j4 test (UNIX), or vcbuild test (Windows) passes
  • [ ] tests and/or benchmarks are included
  • [ ] documentation is changed or added
  • [ ] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

This has some widespread implications:

  • dynamicInstantiate no longer exists since there is no 1st class references between loaders and the thread they are operating on
  • only 1 shared loader is spawned for all the threads it affects, unlike currently where node spins up a new loader on each thread
  • data is done by passing messages which are serialized
  • loaders can no long be affected by mutated globals from non-loader code

This roughly follows some of the older design docs and discussions from @nodejs/modules .

This does not seek to allow having multiple user specified loaders, nor is it seeking to change the loader API signatures, it is purely about moving them off thread and the implications of such.

This does introduce a new type of Worker for loading an internal entry point and also expands the worker_threads API for convenience by allowing a transferList in the workerData to avoid extraneous postMessages.

This will need quite a large writeup on how it works and how data is transferred but this seems a good point to start discussions.

+400 -74

0 comment

11 changed files

pr created time in 2 months

push eventbmeck/node

Bradley Farias

commit sha 937e53c724f031c7723109325f15445a8616b354

wip start fixing tests

view details

push time in 2 months

push eventbmeck/node

Bradley Farias

commit sha 80bb8b7db0db64966c5022a9a4d29b93c9a02dce

wip, worker_threads running

view details

push time in 2 months

issue commentnodejs/node

Worker threads: ability for tooling to pass data to workers and inject NODE_OPTIONS

It seems like the event listener approach would at least allow some censoring/protecting of options? You could use Object.freeze etc on the options?

coreyfarrell

comment created time in 2 months

more