@nodejs/crypto Any other reviews?

Fixes: https://github.com/nodejs/node/issues/34699

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] tests and/or benchmarks are included
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

@juanarbol Fwiw, you’re not the first one who ran into this, so it might be worth adding a comment here that this is intentional (especially considering that this PR already had 4 approvals)

@jetibest Thanks for the comment!

The issue goes away for me when passing the --no-concurrent-array-buffer-sweeping --no-concurrent-array-buffer-freeing flags to Node.js, so I think it’s safe to say that this is a V8 bug.

Could you open a new issue here and on https://bugs.chromium.org/p/v8/issues/ (and mention that setting those flags works around the problem)?

@bridgear No – this is als the cause of the failures in the WASI tests here, I assume.

More generally, in C++

if (x) {
  some_variable a;
  do_something(); 
} else {
  do_something(); 
}

and

if (x) {
  some_variable a;
}
do_something(); 

are not equivalent if the destructor of a influences do_something() in some way (which is the case here).

Hi @calumsult, and thanks for the PR! I have a few questions:

• What advantage does this have over looking up the symbol? Was there any particular problem with the current code?
• How would you like this to be published? We provide prebuilt binaries for ref-napi, and they can only publish one particular version of the addon at a time.
• Publishing a version of ref-napi that links the functions directly would increase the requirements to a version of Node.js that supports N-API 6. That would be a breaking change, to my understanding. Is that worth it?

Yes, looks like at least the ARM build is failing. It’s probably easiest to keep this as-is?

This changes the type here of &time_val.tv_sec from time_t* to int64_t*, which might not be the same pointer types. Isn’t that an issue? (I’ll kick off CI to see if it’s a problem on one of our supported platforms, but obviously there are and have been platforms where time_t is a 32-bit type.)

Some stylistic changes to bring this more in line with what our tests currently look like, and add a note about general flakiness.

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] tests and/or benchmarks are included
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

@mmarchini This would need a manual backport to v14.x, unfortunately

@jasnell This would need a manual backport for v14.x, there’s a bunch of conflicts

On the one hand, the commit message here should be fixed, on the other hand, it should probably also match the one on master – what do we usually do in that type of situation?

We use node (REPL) in one place and Node.js (REPL) in another place in error messages in repl.js. Use Node.js in both places.

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

Landed in 989d82fe013def851f76309bcadd2e8847c8ee73

CallJSOnreadMethod expects the return value to be undefined or a new buffer, so make sure to return nothing, even when an error causes us to destroy the stream.

Fixes: https://github.com/nodejs/node/issues/34346

Checklist

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
  • (except test-benchmark-napi, which has been broken since I cloned)
• [x] commit message follows commit guidelines

• Version: 14.4.0
• Platform: Linux 4.19
• Subsystem: net

occasionally, we see nodejs 14.4.0 drop core with a message like:

E 2020-07-01T21:38:27.737670958Z node[1]: ../src/node_buffer.cc:242:char* node::Buffer::Data(v8::Local<v8::Value>): Assertion `val->IsArrayBufferView()' failed.
I 2020-07-01T21:38:27.737769249Z [20200701-21:38:27.737] TRA conclave-5c84d47bd8-5b4pj (03e741ad-68e6-4f01-939c-922b53fb6004) @52: Socket error: read ECONNRESET
E 2020-07-01T21:38:27.738570272Z  1: 0xa2b020 node::Abort() [node]
E 2020-07-01T21:38:27.739024375Z  2: 0xa2b09e  [node]
E 2020-07-01T21:38:27.739592654Z  3: 0xa037aa node::Buffer::Data(v8::Local<v8::Value>) [node]
E 2020-07-01T21:38:27.740180478Z  4: 0xaf6481 node::CustomBufferJSListener::OnStreamRead(long, uv_buf_t const&) [node]
E 2020-07-01T21:38:27.740705062Z  5: 0xb02058 node::LibuvStreamWrap::OnUvRead(long, uv_buf_t const*) [node]
E 2020-07-01T21:38:27.741487768Z  6: 0x132fb41  [node]
E 2020-07-01T21:38:27.742341825Z  7: 0x1330310  [node]
E 2020-07-01T21:38:27.743122216Z  8: 0x1336c50  [node]
E 2020-07-01T21:38:27.743991594Z  9: 0x13250f8 uv_run [node]
E 2020-07-01T21:38:27.744573560Z 10: 0xa6b9b4 node::NodeMainInstance::Run() [node]
E 2020-07-01T21:38:27.745148797Z 11: 0x9fae81 node::Start(int, char**) [node]
E 2020-07-01T21:38:27.745172061Z 12: 0x7f290cdd22e1 __libc_start_main [/lib/x86_64-linux-gnu/libc.so.6]
E 2020-07-01T21:38:27.745732198Z 13: 0x99529c  [node]

it seems to be always the same location, and always in response to an ECONNRESET on the socket. it only happens when using the "onread" hook in net.Socket.

the good news: after some trial & error, and digging through the nodejs code, i think i figured out why:

when we set "onread", the Socket init sets kBuffer and kBufferCb (net.js:306). then initSocketHandle will call useUserBuffer on the handle (net.js:254). this sets the uv listener to be a CustomBufferJSListener (stream_base.cc:59). so far so good.

when new data arrives, OnStreamRead calls CallJSOnreadMethod (stream_base.cc:535). if the JS method returns a value, it's assumed to be a new buffer to use next time.

the JS method is onStreamRead (stream_base_commons.js:163). if we got data, it returns nothing, unless kBufferGen is set, in which case it fetches a new buffer and returns that. cool.

if we got a non-EOF error, it instead destroys the socket and returns (line 205). unfortunately it uses a shortcut and returns whatever value destroy returns. destroy (destroy.js:5) always returns this. so the Socket gets returned and treated as a new buffer until it fails the assertion in Buffer::Data and crashes.

i think the easiest fix for this is to replace net.js line 205 to not return the socket:

    stream.destroy(errnoException(nread, 'read'));
    return;

i tested this by monkey-patching the socket destroy function to return nothing:

    const old_destroy = socket.destroy;
    socket.destroy = function (...args) { old_destroy.apply(this, args); };

this has been running for 6 days now without a crash, so i think it proves the fix.

Landed in c0be31f0efd162f0edd416ed23f5aac082edd2aa, and sorry for the delay!

Handle situations in which the main Promise from a TLA module is not fulfilled better:

• When not resolving the Promise at all, set a non-zero exit code (unless another one has been requested explicitly) to distinguish the result from a successful completion.
• When rejecting the Promise, always treat it like an uncaught exception. In particular, this also ensures a non-zero exit code.

Refs: https://github.com/nodejs/node/pull/34558

I’m sorry, this is a bit late. I’ve added the dont-land label on #34558 – I remember these points being brought up before in discussion (not sure where), but it seems like they weren’t revisited before unflagging TLA on master. I personally think they need to be addressed before we actually ship this, and in particular silently treating await unresolved or await rejected as a success is not acceptable.

@mylesborins @nodejs/modules-active-members

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] tests and/or benchmarks are included
• [x] documentation is changed or added
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

Landed in e948ef351b4111d78ae8ba08f29eb138af22fa0b

deps: V8: cherry-pick e06ace6b5cdb

Original commit message:

[api] Fix empty Maybe crash in GetRealNamedPropertyAttributes

`Object::GetRealNamedPropertyAttributes()` can crash if an empty
`Maybe` is returned by `JSReceiver::GetPropertyAttributes()` because
it was not checking for that. Fix that.

Refs: https://github.com/nodejs/node/issues/34606
Change-Id: Ic83f904ba7134786bcd8f786eb2ce98adb4fea1e
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2335057
Commit-Queue: Leszek Swirski <[email protected]>
Reviewed-by: Leszek Swirski <[email protected]>
Cr-Commit-Position: refs/heads/master@{#69258}

Refs: https://github.com/v8/v8/commit/e06ace6b5cdb64b6abfe8e1229f2159b7a38b4e7

test: add vm crash regression test

Fixes: https://github.com/nodejs/node/issues/34606

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] tests and/or benchmarks are included
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

@jasnell No, and it’s odd that it’s only failing on ARM… that might be a V8 bug?

Fixes: https://github.com/nodejs/node/issues/34657 Refs: https://github.com/nodejs/node/pull/34572

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

I don’t think using a default like that would be helpful in most situations

@jasnell The previous entry was perfectly fine – I assume this was intentional, we do follow this format for cases in which semver-minor PRs get backported so that we can list multiple versions

I can do a manual backport

This seems like it should be fairly uncontroversial, :+1: to fast-tracking?

To be blunt, I don’t think we can get any kind of reasonable async tracking for event emitters that isn’t implemented by the built-in EventEmitter class itself, there’s just too much code written out there that doesn’t expect this. AsyncResource.bind or a possible asyncResource option to the EventEmitter constructor (which probably wouldn’t even give us the right thing!) are nice but just not practical as actual solutions in the bigger picture here.

And I know it’s not great to go and decrease EE performance, but if we end up recommending everybody to do the changes on the consumer side, the only difference is the massive amount of work that needs to be done in the ecosystem.

• Version: v14.6.0, v12.14.1
• Platform: 4.15.0-1091-oem #101-Ubuntu SMP Thu Jun 25 17:55:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: worker_threads

What steps will reproduce the bug?

I have opened up a repository here that can be used to reliably crash node. Follow the steps in README.md to replicate.

How often does it reproduce? Is there a required condition?

This reproduces so long as a native node module has an active async handle for async completion.

What is the expected behavior?

Worker thread should not terminate until all handles are cleaned up. Modules should be able to exit gracefully when terminate is requested.

npm test

> [email protected] test /home/tylerw/github/test/native-node-tests
> node -e "w = new (require('worker_threads').Worker)('./runDoAsyncThing.js'); setTimeout(w.terminate.bind(w), 2000);"

Entering execute thread

What do you see instead?

npm test

> [email protected] test /home/tylerw/github/test/native-node-tests
> node -e "w = new (require('worker_threads').Worker)('./runDoAsyncThing.js'); setTimeout(w.terminate.bind(w), 2000);"

Entering execute thread
uv loop at [0x7f18151e1ad8] has open handles:
uv loop at [0x7f18151e1ad8] has 0 open handles in total
node[23076]: ../src/debug_utils.cc:322:void node::CheckedUvLoopClose(uv_loop_t*): Assertion `0 && "uv_loop_close() while having open handles"' failed.
 1: 0x9fb000 node::Abort() [node]
 2: 0x9fb07e  [node]
 3: 0x98e9f1  [node]
 4: 0xab2e44 node::worker::Worker::Run() [node]
 5: 0xab2e88  [node]
 6: 0x7f1817de06db  [/lib/x86_64-linux-gnu/libpthread.so.0]
 7: 0x7f1817b09a3f clone [/lib/x86_64-linux-gnu/libc.so.6]
Aborted (core dumped)
npm ERR! Test failed.  See above for more details.

Sometimes addons need to perform cleanup actions, for example closing libuv handles or waiting for requests to finish, that cannot be performed synchronously.

Add C++ API and N-API functions that allow providing such asynchronous cleanup hooks.

Fixes: https://github.com/nodejs/node/issues/34567

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

@nodejs/n-api

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] tests and/or benchmarks are included
• [x] documentation is changed or added
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

Landed in 22cbbcf9d9374d4b663bf1409f292212fa57623a

SA_SIGINFO should always be set, regardless of reset_handler, right?

  if (!(max >= min)) {
    throw new ERR_OUT_OF_RANGE('max', `>= ${min}`, max);

Since the ranges are inclusive now? It doesn’t make a lot of sense to explicitly specify two equal values at first sight, but the logical bound to require here is max >= min, not max > min

  - REPLACEME

(see the earlier comment – REPLACEME is a special string, other values will not be picked up during the release process)

    throw new ERR_OUT_OF_RANGE('max - min', `< ${RAND_MAX}`, range);

@JJBocanegra As @devsnek said, this is a bug in your code. It’s not really related to this issue here.

@JJBocanegra Can you open a new issue at https://github.com/nodejs/help/issues/ instead of discussing this one?

I don’t think most people would know what “ESM Import Stalled” means.

Again, I think this is mostly relevant when you’re actually developing a loader or similar, and in that case you most likely don’t even need any explanation at all.

Only that :)

To be clear, I consider any kind of failure to finish the main module’s evaluation, whether through TLA or not, something that should not be considered a successful operation. This is intentional, and I would be against restricting the check here to only module.evaluate().

Hm … for some reason I thought I saw an UnhandledRejectionWarning from TLA when trying it out, but you should be right, yes. Not sure what’s up with that, but I’ve updated it to keep using that :+1:

Are you suggesting to explicitly list that special case? I would prefer not to, if you’re working on an ESM loader then you can probably imagine that import() stalling would look like unfinished TLA. The far more common case is going to be actual TLA usage, I assume

@codebytere I don’t know – is there a practical difference? The next step would be to just remove the label again once #34640 lands, regardless of which one is used, right? Anyway, feel free to put the correct one here if I got it wrong. :)

Just so the context is here as well, I’ve added dont-land-on-v14.x because I think that this should not be released without #34640, and dont-land-on-v12.x because my understanding is that it wouldn’t apply there.

@MylesBorins In my very subjective opinion, this PR would bring things where they need to be. ;)

Handle situations in which the main Promise from a TLA module is not fulfilled better:

• When not resolving the Promise at all, set a non-zero exit code (unless another one has been requested explicitly) to distinguish the result from a successful completion.
• When rejecting the Promise, always treat it like an uncaught exception. In particular, this also ensures a non-zero exit code.

Refs: https://github.com/nodejs/node/pull/34558

I’m sorry, this is a bit late. I’ve added the dont-land label on #34558 – I remember these points being brought up before in discussion (not sure where), but it seems like they weren’t revisited before unflagging TLA on master. I personally think they need to be addressed before we actually ship this, and in particular silently treating await unresolved or await rejected as a success is not acceptable.

@mylesborins @nodejs/modules-active-members

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

• [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
• [x] tests and/or benchmarks are included
• [x] documentation is changed or added
• [x] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

@simonschick I’m not sure I’ll have time for finishing that task up this week.

So can you pick either one of your review comments? You’ve asked me to bump the version, and subsequently asked me to bump the code so that it matches that version; that basically leaves me with no option but a massive scope creep for this PR. I don’t think that’s a helpful review process, and I would appreciate it if you were to pick one (update the version or not, but not expect me to do a ton of extra work).

the pull request**. If reaching consensus is not possible, a Collaborator may

Our options parser natively supports passing an option multiple times, e.g. -u condA -u condB or --conditions condA --conditions=condB. Using that tends to be a bit nicer, because it means that one piece of code (in a tool that spawns a Node.js process) can set a flag without worrying about whether another piece of code has set the same flag already.

Specifying the type of the option variable as std::vector<std::string> instead of a plain std::string is sufficient for that.

(In that case it might make sense to name the option --condition, i.e. singular)

@iamstarkov Right, but graceful-fs (which is not just a dependency of npm, it’s actually shipped as part of the npm package) depends on Node.js internals in a way that does not follow semver. In particular, in this case, we started using a custom option for a file system operation inside of Node.js itself, and the monkey-patched fs module from graceful-fs did not recognize that option in earlier versions.

Done in v2.0.4!

<!-- Thank you for reporting an issue.

This issue tracker is for bugs and issues found within Node.js core. If you require more general support please file an issue on our help repo. https://github.com/nodejs/help

Please fill in as much of the template below as you're able.

Version: output of node -v Platform: output of uname -a (UNIX), or version and 32 or 64-bit (Windows) Subsystem: if known, please specify affected core module name -->

• Version: 14.7.0
• Platform: Darwin Simens-MacBook-Pro.local 18.7.0 Darwin Kernel Version 18.7.0: Thu Jun 18 20:50:10 PDT 2020; root:xnu-4903.278.43~1/RELEASE_X86_64 x86_64
• Subsystem: N-API (I think)

What steps will reproduce the bug?

Clone https://github.com/node-ffi-napi/weak-napi, run install and run the tests. They fail on node 14.7.0, but pass on node 14.6.0.

I discovered this via Jest's tests failing (which use weak-napi). I assume Jest's --detect-leaks feature is broken for this version of Node.

(weak-napi could probably be added to CITGM)

How often does it reproduce? Is there a required condition?

It always fails

What is the expected behavior?

Tests should pass 🙂

<!-- If possible please provide textual output instead of screenshots. -->

What do you see instead?

Tests fail 🙁

<!-- If possible please provide textual output instead of screenshots. -->

Additional information

I haven't bisected, but #34386 seems like the obvious candidate looking at the changelog

/cc @addaleax

<!-- Tell us anything else you think we should know. -->

Yeah, unless this is causing any trouble besides the timing difference, I think there’s nothing actionable here. Let us know if we should reopen :)

I'm trying to debug a crash of my application. (Not isolated yet)

When reading the source code, it seems the following function may write over the end of buffer, did I miss anything?

L582 may return a len = size which equal to buffer.length-offset. so at L583, offset+len equal offset+buffer.length-offset = buffer.length

Shouldn't the L581 be const size = buffer.length - offset-1?

https://github.com/node-ffi-napi/ref-napi/blob/7d0d7b24d242372c7557d571cbe2a87ef67ac53d/lib/ref.js#L572-L584

Fwiw, the line in question does not actually perform an OOB write; it throws an exception if offset+len is OOB, so it’s probably not the cause of the crash you were seeing.

Thanks for pointing this out! Yes, this looks like a bug. I’ll publish a new release shortly.

@simenb There isn’t really any strong guarantee, partly because V8 itself also doesn’t give us any strong guarantees. 2 × setImmediate is currently enough, so you can pick 10 or so if you want to be reasonably safe (that is, as long as setImmediate() works at all).

I haven't bisected, but #34386 seems like the obvious candidate looking at the changelog

I’ve confirmed that reverting it fixes the weak-napi test suite, yes.

(weak-napi could probably be added to CITGM)

@nodejs/citgm I’d be :+1: on this.

That being said, I don’t think the weak-napi breakage qualifies as a bug in Node.js or N-API. The weak-napi tests are/were simply expecting stricter relative timing guarantees than what Node.js provides, and fixing the tests up (node-ffi-napi/weak-napi@fdafbdefcfe1a418ddae0cca36974a401f4ec7ef) seems like the right thing to do here, at least with my N-API and weak-napi maintainer hat on.

@iamstarkov The problem is the [email protected] (hardcoded in your package.json, no semver range) → @semantic-release/npm@^5.2.0-beta.5 → [email protected] (also hardcoded, no semver range) dependency chain. Simply upgrading your dependencies should be enough. You may not want to rely on beta versions of tools anyway.

@0ams Yes, this should also be part of the next v12.x release, I think.

final for all of these?

This should report memory usage

These should go into the private: section as implementation details, imo?

Would it possibly make more sense to remove the O_TRUNC flag from here?

Wouldn't that break copy operations that overwrite existing files?

We could manually truncate using ftruncate() after fstat() on the already-opened fd, right?

@gengjiawen Sorry, yes – do you want to open a PR to fix this?

Tbh, I’d personally prefer consistency with randomBytes(). We can totally add crypto.promises like we did with fs/dns.

Perhaps an inbuilt snapshot is used to speed up initialization? (Yocto configures Node with --without-snapshot..

That sounds like a likely candidate, yes.

@gabrielschulhof I’ve converted this to a C file.

However, I would strongly encourage you to address this yourself for other tests instead of expecting people to do the work as part of the PR review process. It’s generally not a great situation for contributors if you can’t base a new test off existing tests.

Fwiw, generally the terms “minimum” and “maximum” are considered to be inclusive. However, I don’t know of a good term for the upper bound that would be implicitly exclusive, so I hadn’t commented before here. start and end might be better, comparable to the behavior for e.g. array functions like .slice().

Opened https://chromium-review.googlesource.com/c/v8/v8/+/2335057 to address this in V8

@iandrc That depends on the commit, e.g. whether it’s partially backportable or not. But probably leave it as-is, yes.

@iandrc Sure – just be prepared, I think this may be something that might intentionally have been changed as a semver-major commit (i.e. not backportable).