profile
viewpoint
Gireesh Punathil gireeshpunathil IBM India Working to improve man - machine interactions

expressjs/discussions 27

Public discussions for the Express.js organization

gireeshpunathil/nodev0.12.7tlsmemoryleak 2

Standalone test case which shows C++ objects pertinent to TLS/SSL leaking over time

gireeshpunathil/nodev0.10httpsmemoryleak 1

Contains test case which demonstrates how Buffer, Request, IncomingMessage objects are leaking across completely recycled transactions.

gireeshpunathil/admin 0

Facilitating joint collaboration amongst the TSC and CommComm

gireeshpunathil/appsody 0

Appsody command line tool.

gireeshpunathil/appsody-buildah 0

A docker image with Appsody CLI installed that can be used for running Appsody with buildah in Tekton pipelines.

gireeshpunathil/appsody-docker 0

A docker image with Appsody CLI installed. Useful for running Appsody in Tekton pipelines.

gireeshpunathil/blink 0

Example file to blink the LED on an Arduino

gireeshpunathil/build 0

Better build and test infra for Node.

issue commentnodejs/help

cant run node.js on mac plz help

bye default core files go to /cores/ in mac

ItsACodeWorld

comment created time in 12 hours

issue commentnodejs/help

cant run node.js on mac plz help

@ItsACodeWorld - could you try after setting this: $ ulimit -c unilmited

so that it produces a core dump?

and then you can launch the dump in a debugger and collect the stack trace:

$ lldb <path-to-node> <path-to-core>
(lld) bt
ItsACodeWorld

comment created time in 12 hours

pull request commentnodejs/node

src,lib: don't run bootstrapper in CreateEnvironment

where is the other bootstrapping that is happening?

codebytere

comment created time in 15 hours

pull request commentnodejs/node

doc: Add more useful information to Glossary.md

one advantage of this approach (as opposed to linking to external sites) is, meaning can be interpreted in our usage context. example: https://github.com/nodejs/node/blob/0c3c0e7184c4ca625148c14d61d8e9438a1b69f2/src/node_version.h#L78-L80

here, we are not really using this term to refer to hardware architectures or subroutine linkage conventions; but to a protocol by which node interacts with the the dependent modules? so instead of taking to the general literature of its definition, history etc. a short description that will satisfy the need for some one reading through the code will be more effective?

another example: https://github.com/nodejs/node/blob/0c3c0e7184c4ca625148c14d61d8e9438a1b69f2/src/node_options.cc#L541-L544

here, the reader may not want to know the complete story of https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop - as at that point they may not be interested in that; but just to fill the gap of their understanding (a simple expansion) and move on with the option parsing?

we do have precedence here in https://github.com/nodejs/node/blob/0c3c0e7184c4ca625148c14d61d8e9438a1b69f2/CPP_STYLE_GUIDE.md as well as https://github.com/nodejs/node/blob/0c3c0e7184c4ca625148c14d61d8e9438a1b69f2/src/README.md that provide customized knowledge on C++ usage in the code base.

and technically too I find this approach correct - sticking to the definition of glossary: a brief dictionary of used words that are uncommon.

HarshithaKP

comment created time in a day

pull request commentnodejs/node

cli: --perf-prof only works on Linux

ok, thanks @mmarchini, that will really help!

for the context, my use case is here: https://github.com/nodejs/diagnostics/tree/master/documentation/profiling - attempt to document diagnostic best practices around profiling. These are aimed at user-journey driven, endorsed by node project, and planned to be supported for long term.

as that is a conversation with v8, and this PR is just documenting the code behavior that was uncovered in the field, it looks good to me!

codebytere

comment created time in 2 days

pull request commentnodejs/node

cli: --perf-prof only works on Linux

they (v8) did have a (documented?) promise that they would consider Node's compatibility as a major factor while implementing features?

codebytere

comment created time in 2 days

issue commentnodejs/node

How much is my memory?

@bnoordhuis - unfortunately, the LD_PRELOAD approach does not help on the immediate issue (#23277) or the wider need (accurate metering in Cloud deployments)

An in-built, callable API will be the best way, and for the long term.

gireeshpunathil

comment created time in 3 days

Pull request review commentnodejs/diagnostics

doc: add --trace-sigint diagnosing document

-// TODO+# Using SigInt Traces for Call Stacks++`--trace-sigint` CLI option will print a message and stack traces on SIGINT.+It can be convenient to find out what's the process doing when using shells+launched the Node.js processes.++> Caveat:+> `--trace-sigint` CLI option is available since Node.js v13.9.0.

should we add another caveat that says the process control behavior of SIGINT is un-altered otherwise? i.e, by default, the process also get terminated after the print?

legendecas

comment created time in 3 days

issue commentnodejs/node

How much is my memory?

@bnoordhuis - ok, but Libuv has malloc wrappers that can potentially keep a track of active allocations?

https://github.com/libuv/libuv/blob/18ff13c71e5e68066729bc90664f9550bfe5ead4/src/uv-common.c#L75-L79

I am not saying the entire app's memory demand funnel through Libuv. My take is that given Node.js is a language runtime that abstracts many implementation details from the underlying system, one or more or all of libuv / v8 / node should be able to measure the real memory allocation either pro-actively (through malloc wrapping) or reactively (through relecting on process's memory, with help of underlying platform if need be).

In the absence of that, the closest what we get is rss and virtual mem. As illustrated in the linux example, those are too far from the real memory (96 instead of 12) to be of any meaningful accounting purpose?

gireeshpunathil

comment created time in 3 days

issue commentnodejs/node

Segmentation fault in taking a heap snapshot

few instructions before the RIP pls? (x/10i 0x000000000095ff80)

lomaster1

comment created time in 4 days

issue commentnodejs/node

How much is my memory?

@bnoordhuis - malloc cache is a term I coined, for lack of a better qualifier. If you look at my calculation in the first comment, 94 MB is seen as allocated to the process, after the malloc/free business is long over - out of which only 12 MB belongs to the program, the rest has to be released and un-accounted from virtual memory, but it hasn't - so my guess is that it is cached by malloc subsystem, for future requests.

How would you measure that? The approach in Gregg's article is... creative.... but not something that can be generalized across platforms, possibly not even architectures.

I agree that it is complicated, and might need OS support to achieve that; and hence I realistically opted for the first one.

gireeshpunathil

comment created time in 4 days

issue commentnodejs/node

child_process execFile hangs after execution

unable to reproduce.

can you produce a report through Analyze Wait Chain feature? (task manager -> select the parent node process -> right click -> analyze wait chain)

cassiourugit

comment created time in 4 days

issue commentnodejs/node

How much is my memory?

with that, I would like to convert this into a (libuv?) feature request:

use case: ability for applications to figure out its memory usage in a reasonable manner - one of:

  • virtual memory minus chunks residing in malloc cache
  • working set size: memory which the app is actually engaged in

with a preference on the former.

/cc @nodejs/libuv affected module: @nodejs/workers

gireeshpunathil

comment created time in 4 days

issue commentnodejs/node

How much is my memory?

I tested in mac and windows, and saw the same behavior - with minor differences w.r.t numbers (probably owing to the system configurations and implementation differences in memory management at glibc / kernel), but the pattern is same.

I found references that confirm my theory: http://www.brendangregg.com/wss.html https://www.networkworld.com/article/2298360/kernel-space--how-much-memory-am-i-really-using-.html

relevant excerpt:

The problem is that the numbers exported by the current kernels are nearly meaningless. 
The reported virtual size of an application is nearly irrelevant; it says nothing about 
how much of that virtual space is actually being used. The resident set size (RSS)
number is a little better, but there is no information on sharing of pages there. The 
/proc/pid/smaps file gives a bit of detail, but also lacks sharing information. And the
presence of memory pressure can change the situation significantly.
gireeshpunathil

comment created time in 4 days

issue commentnodejs/diagnostics

reportVersion semantics are not defined

Will a format change of string types be counted as a significant change? Like the format of error stacks.

@legendecas - I don't think so, as this sort of change implies no change in the report's functionality, but a change in node core w.r.t data representation.

addaleax

comment created time in 5 days

issue commentnodejs/diagnostics

reportVersion semantics are not defined

makes sense @mhdawson . so the modified proposal is:

a single digit versioning with version bump on every structural changes wherein structural change is anything that:

  • adds a new key
  • removes a key
  • changes the data type for a field
addaleax

comment created time in 5 days

issue commentnodejs/diagnostics

reportVersion semantics are not defined

We deliberated in the last(-to-last) WG meeting on this.

Report being a data (as opposed to code), patch does not make sense anyways. So the contention is really between x.y semantics versus x semantics.

I propose a single digit versioning with version bump on every structural changes wherein structural change is anything that:

  • adds a new key
  • removes a key (though chances are less for this to happen)

It is reasonable to expect report parsers to parse it in JSON-native way, instead of line-parsing, char-parsing etc. One of the main motivation for the report to be in JSON format was easy-parsing for consuming tools. Because of this, section movements / aesthetic modifications do not cause a version bump.

It is reasonable to expect no data type changes being applied to the fields.

addaleax

comment created time in 5 days

issue closednodejs/node

Wanna see my hot pics?💋💋💋

Hi!😻😻😻 Wanna see my hot pics?💋💋💋 Go to this site and find me here. ❤️❤️❤️ My nick is sexygirl662 👇👇👇

https://tinyurl.com/sbmadwq

closed time in 5 days

akdnfbf

pull request commentnodejs/node

doc: move gireeshpunathil to TSC emeritus

@mhdawson @Trott - do you know if there are other formalities that required to complete the formalities here - such as removing from email listings, tsc meeting lists, org-wide elevated privileges, etc. ?

gireeshpunathil

comment created time in 6 days

PR closed nodejs/node

doc: move gireeshpunathil to TSC emeritus author ready doc

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

  • [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
  • [ ] tests and/or benchmarks are included
  • [x] documentation is changed or added
  • [x] commit message follows commit guidelines

to meet the TSC membership guidelines in terms of employer representation distribution in the team.

blocked on: https://github.com/nodejs/node/pull/31725 Fixes: https://github.com/nodejs/TSC/issues/817

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

+2 -2

2 comments

1 changed file

gireeshpunathil

pr closed time in 6 days

pull request commentnodejs/node

doc: move gireeshpunathil to TSC emeritus

landed in 4c746a6cfda980c1cd0de6246781c0083d9e416c

gireeshpunathil

comment created time in 6 days

push eventnodejs/node

Gireesh Punathil

commit sha 4c746a6cfda980c1cd0de6246781c0083d9e416c

doc: move gireeshpunathil to TSC emeritus PR-URL: https://github.com/nodejs/node/pull/31770 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Myles Borins <myles.borins@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

view details

push time in 6 days

PR closed nodejs/node

Readme updates fishrock123 author ready doc

It was a good run. Almost 5 years.

I haven't really been involved in the last 3+? months though, so it's time I call it and 'retire' from the TSC.

I think it is unlikely that I'll be on the TSC again, as node is unfortunately becoming increasingly disinteresting (& frustrating) to me.

Also moves me to a past releaser, as I haven't done any in the past 1-2 year(s) I don't really plan on doing more.

(So long and thanks for all the fish!)

(I am aware there is a list of other access that will need to be cleaned up...)

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

+5 -6

4 comments

1 changed file

Fishrock123

pr closed time in 6 days

pull request commentnodejs/node

Readme updates fishrock123

landed in 4d6c861...30e6049

Fishrock123

comment created time in 6 days

push eventnodejs/node

Jeremiah Senkpiel

commit sha 4d6c861800cc6ac1dc6a0fd9d3a8b0053baec62a

doc: move @Fishrock123 to a previous releaser I have not done a release in well over a year, maybe even two. I also don't really plan to do more, as Node.js releases are very tedious. PR-URL: https://github.com/nodejs/node/pull/31725 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Myles Borins <myles.borins@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

view details

Jeremiah Senkpiel

commit sha 928c210a611fb6b4575843708e7cb921f311ebde

doc: move @Fishrock123 to TSC Emeriti It was a good run. Almost 5 years. I haven't really been involved in the last 3+? months though, so it's time I call it and 'retire'. I think it is unlikely that I'll be on the TSC again, as node is unfortunately becoming increasingly disinteresting (& frustrating) to me. (So long and thanks for all the fish!) PR-URL: https://github.com/nodejs/node/pull/31725 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Myles Borins <myles.borins@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

view details

Jeremiah Senkpiel

commit sha 30e6049c75590dec8d5edbf033edbc20e79efcb7

doc: pronouns for @Fishrock123 might as well while I'm at it feels a bit weird being the first person on this list with '/they' but I guess someone's gota do it PR-URL: https://github.com/nodejs/node/pull/31725 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Daniel Bevenius <daniel.bevenius@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Myles Borins <myles.borins@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Anto Aravinth <anto.aravinth.cse@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Tobias Nießen <tniessen@tnie.de> Reviewed-By: Gus Caplan <me@gus.host> Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>

view details

push time in 6 days

pull request commentnodejs/node

[WIP] doc: move gireeshpunathil to TSC emeritus

CI-lie: https://ci.nodejs.org/job/node-test-pull-request-lite-pipeline/4129/

gireeshpunathil

comment created time in 6 days

pull request commentnodejs/node

Readme updates fishrock123

Ci-lite: https://ci.nodejs.org/job/node-test-pull-request-lite-pipeline/4128/

Fishrock123

comment created time in 6 days

create barnchgireeshpunathil/node

branch : move-emeritus

created branch time in 9 days

issue commentnodejs/TSC

Timing of TSC member addition / resignation

I had word with @sam-github . I will go emeritus to address the situation. The WIP PR is in, just waiting for https://github.com/nodejs/node/pull/31725 to be landed.

tniessen

comment created time in 9 days

PR opened nodejs/node

[WIP] doc: move gireeshpunathil to TSC emeritus

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

  • [x] make -j4 test (UNIX), or vcbuild test (Windows) passes
  • [ ] tests and/or benchmarks are included
  • [x] documentation is changed or added
  • [x] commit message follows commit guidelines

to meet the TSC membership guidelines in terms of employer representation distribution in the team.

blocked on: https://github.com/nodejs/node/pull/31725 Fixes: https://github.com/nodejs/TSC/issues/817

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

+2 -2

0 comment

1 changed file

pr created time in 9 days

issue commentnodejs/diagnostics

Node.js Foundation Diagnostics WorkGroup Meeting 2020-02-12

I am not working this week, so wont make to this instance!

mhdawson

comment created time in 11 days

issue commentnodejs/node

assertion failure in src/node_worker.cc with large number of workers

@addaleax - that is really promising! my copy does not have it; will check and confirm. [ since the recreate is consistent, the validation will be easy ]

gireeshpunathil

comment created time in 17 days

issue commentnodejs/node

assertion failure in src/node_worker.cc with large number of workers

On the other hand, in its current form, #31621 will address this one too?

gireeshpunathil

comment created time in 17 days

issue commentnodejs/node

assertion failure in src/node_worker.cc with large number of workers

(gdb) where
#0  0x00007f92514c7377 in raise () from /lib64/libc.so.6
#1  0x00007f92514c8a68 in abort () from /lib64/libc.so.6
#2  0x00007f92514c0196 in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007f92514c0242 in __assert_fail () from /lib64/libc.so.6
#4  0x0000000000a03319 in uv__close_nocheckstdio (fd=-24) at ../deps/uv/src/unix/core.c:556
#5  0x00000000013cd271 in uv__close_nocheckstdio (fd=fd@entry=-24) at ../deps/uv/src/unix/core.c:563
#6  0x00000000013de2f7 in uv__read_proc_meminfo (what=what@entry=0x20be946 "MemTotal:")
    at ../deps/uv/src/unix/linux-core.c:1016
#7  0x00000000013df7f3 in uv_get_total_memory () at ../deps/uv/src/unix/linux-core.c:1043
#8  0x0000000000a0dc05 in node::SetIsolateCreateParamsForNode(v8::Isolate::CreateParams*) ()
#9  0x0000000000b3a3e7 in node::worker::Worker::Run() ()
#10 0x0000000000b3b930 in node::worker::Worker::StartThread(v8::FunctionCallbackInfo<v8::Value> const&)::{lambda(void*)#1}::_FUN(void*) ()
#11 0x00007f9251866ea5 in start_thread () from /lib64/libpthread.so.0
#12 0x00007f925158f8cd in clone () from /lib64/libc.so.6

this is the stack trace for the second assertion. Looks like the loop creation failed, but the worker creation sequence has progressed thus far, wrongly?

when we run out of descriptors, the error is -24, which is returned by uv_loop_init. the fd at the assertion site is also the same. Is it by chance? or do we fill the uv_loop_t with the error codes?

gireeshpunathil

comment created time in 17 days

issue closednodejs/help

Clarify example for AsyncResource

<!-- Please remove this line and fill the template -->

  • Node.js Version: 10
  • OS: N/A
  • Scope (install, code, runtime, meta, other?): ?Documentation
  • Module (and version) (if relevant): async_hooks

Hi there,

Can someone clarify the purpose of the new AsyncResource class? The example provided doesn't seem to show why it is beneficial.

Thanks, Dan

closed time in 17 days

daprahamian

issue commentnodejs/help

Clarify example for AsyncResource

closing this as https://github.com/nodejs/node/pull/31601 has landed that covers this.

daprahamian

comment created time in 17 days

issue commentkabanero-io/kabanero-pipelines

Bug: performance problem in build-bud step of nodejs and nodejs loopback pipeline

@jdmcclur - one or more of:

  • the tests we ran are different
  • the systems we used may have different fuse-overlay versions

We are in the process of:

  • figuring out which RHEL versions match fedora 30/31 where kernel supports new fuse
  • figuring out earliest known good version of fuse
  • figuring out the right steps to upgrade fuse
  • figuring out which test failures map to fuse issue, and which are not.
tam512

comment created time in 18 days

issue openednodejs/node

How much is my memory?

<!-- Thank you for reporting a possible bug in Node.js.

Please fill in as much of the template below as you can.

Version: output of node -v Platform: output of uname -a (UNIX), or version and 32 or 64-bit (Windows) Subsystem: if known, please specify the affected core module name

If possible, please provide code that demonstrates the problem, keeping it as simple and free of external dependencies as you can. -->

  • Version: master
  • Platform: linux / all
  • Subsystem: process

<!-- Please provide more details below this comment. -->

This is opened for 3 reasons:

  • help progress #23277
  • gather intelligence on internal memory management where I may have lack of clarity
  • help clearly define process's memory usage - for future support and documentation (specifically for cloud workloads where usage has bearings on billing)

$ cat foo.cc

#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <malloc.h>

void *list[1024 * 1024];

int r() {
  // add a one so as to never return 0
  return 1 + random() % 137;
}

int main() {
  // use calloc so that the pages are 'touched'.
  for(int i=0;i< 1024 * 1024;i++)
    list[i] = calloc(r(), 1);
  for(int i=0;i< 1024 * 1024;i++)
    free(list[i]);
  sleep(60 * 1000);
}

A linux (RHEL 8) observation: The static memory usage (without the callocs) of this code is 12 MB

  • 8 MB for the array (64-bit, so 8 * 1MB)
  • 4 MB for a.out + libc + ...

top output after the program runs and sleeps for enough time:

$ top -p 6777
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                            
 6777 root      20   0   99772  96344    932 S   0.0   5.1   0:00.16 a.out 

$ cat /proc/meminfo | grep MemFree
MemFree:         1637932 kB
$

top output after the system is heavily loaded:

top -p 6777
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                            
 6777 root      20   0   99772      0      0 S   0.0   0.0   0:00.16 a.out 
$ cat /proc/meminfo | grep MemFree
MemFree:           70444 kB

pmap shows 85 MB is still mapped into the process, matching with top.

0000000002c7c000  87384K rw---   [ anon ]

Observations:

  • rss stays high (~ virtual memory) for ever, if there is no load in the system
  • virtual memory stays high even after all the allocated memory is freed.
  • all numbers tally: 85 for the dynamic memory + 12 for the static part = 97, as shown in top

Problems / questions:

  • what is a meaningful measure of program's memory usage? we can say these are the values we get from the OS, and leave it for the user to interpret those, but that does not help them enough. rss is a function of not only on the program's activity with memory, but also to other parts of the system. virtual memory on the other hand was apparently the sum of allocations (malloc/calloc/mmap...) in the program, but looks like it is not, and depends on the glibc's memory management implementations and its discretion as to when to retain and when to release chunks back to OS, and as to what size of chunks qualify for removal, and may be also depend on the memory usage hueristics as well! When I used 4K chunks for calloc/free, I got different results.

In Cloud workloads this has implications. A user would want to minimize memory tagged against their process and would do everything in their application to achieve that, but due the dependency on the system load and allocation caching behavior, this would not translate well?

  • memory leak definition becomes loose and detection becomes less comprehensive. Plurality of tools profiling the app from different angles would bring-in mutually contradicting results? For example: in a batch process, a code that genuinely leaks an MB versus an MB that is cached in malloc arenas - what is the difference from outside, and why should I worry more about the former and not worry about the later?

  • I read about wss (working set size) - that is a true measure of how much memory the program is actively engaged in. The idea looks like tracking paging-in; as that would be a true reflection of the program's needs. I don't find an API for this, nor a platform-neutral way to obtain this. Even if we could, this has an issue - the value is transient, as it depends on the history of usage, not the current state?

created time in 18 days

IssuesEvent

issue commentnodejs/node

Investigate flaky parallel/test-worker-memory

ok, I am able to confim my theory. I found a system where I can consistently reproduce this failure.

looking at the system spec w.r.t cpu and memory:

$ cat /proc/cpuinfo | grep processor | wc -l
120
$ cat /proc/meminfo | grep MemFree
MemFree:        1041442704 kB

in view of the rss theory, the issue now become evident - there was no drivers for pulling down the bloated rss.

diff --git a/test/parallel/test-worker-memory.js b/test/parallel/test-worker-memory.js
index 19b89d4..9d47271 100644
--- a/test/parallel/test-worker-memory.js
+++ b/test/parallel/test-worker-memory.js
@@ -38,7 +38,10 @@ for (let i = 0; i < numWorkers; ++i) {
   run(60 / numWorkers, () => {
     console.log(`done() called (finished=${finished})`);
     if (++finished === numWorkers) {
-      const finishStats = process.memoryUsage();
+      setInterval(() => {
+        const finishStats = process.memoryUsage();
+        console.log(Math.round(finishStats.rss / (1024 * 1024)))
+      }, 2000)
       // A typical value for this ratio would be ~1.15.
       // 5 as a upper limit is generous, but the main point is that we
       // don't have the memory of 50 Isolates/Node.js environments just lying

With this patch I kept the program alive, ran some load on the system, and saw the rss coming down.

I guess we need to devise a new stratgey for checking memory leak in this test case, without the usage of rss.

Re-opening this based on root cause identified.

Trott

comment created time in 19 days

issue commentappsody/stacks

Nodejs-Express stack build gives error `error building at STEP "RUN yum upgrade`

Buildah] useradd: /etc/passwd.323: lock file already used
[Buildah] useradd: cannot lock /etc/passwd; try again later.

this is the where this issue is root caused. I am unable to see this error in https://github.com/appsody/appsody-buildah/issues/10 ?

aadeshpa

comment created time in 19 days

issue openedopenshift/origin

diagnotics operator

Because there are many repositories under this org, I am not sure this is the right one to raise this issue. I did a scan through of most, but finally decided this is the right one. If not, please transfer appropriately, thanks in advance!

Problem: Default (cluster-supplied or vendor-supplied) dump / trace / monitor data not enough for all diagnostic purposes

Observation: Monitors tend to specialize on higher levels of abstractions for collecting and visualizing data. There is always a gap between what they collect and what the platform / runtime is capable of producing. Devops capabilities are unable to leverage the runtime diagnostic tracing / profiling / monitoring functions.

Examples:

  • for a java runtime, there are VM support for method traces, gc traces and JIT traces
  • for node.js runtime, there are VM support for CPU profiling, gc traces and async traces
  • in addition, there are runtime specific tunables for dump (system, heap, snap) collection

Proposal: Implement a diagnostics operator that is aware of the underlying runtime's diagnostic characteristics and features, and can invoke those and collect artifacts on demand.

  • define a uniform nomenclature for diagnostic operations and artifacts across supported runtimes
  • abstract those definitions into standard verbs / resource definitions.
  • these verbs are understood and implemented by operator. Based on the hosted runtime, they trigger corresponding actions

Value:

  • improved manageability of the hosted runtime through the cluster
  • simplified day two operations and advanced problem determination
  • parity with traditional deployments with respect to serviceability

created time in 20 days

issue commentnodejs/node

assertion failure in src/node_worker.cc with large number of workers

(gdb) where
#0  0x00007ffff6e04377 in raise () from /lib64/libc.so.6
#1  0x00007ffff6e05a68 in abort () from /lib64/libc.so.6
#2  0x0000000000a9dea1 in node::Abort() ()
#3  0x0000000000a9df17 in node::Assert(node::AssertionInfo const&) ()
#4  0x0000000000b3ad12 in node::worker::Worker::Run() ()
#5  0x0000000000b3b880 in node::worker::Worker::StartThread(v8::FunctionCallbackInfo<v8::Value> const&)::{lambda(void*)#1}::_FUN(void*) ()
#6  0x00007ffff71a3ea5 in start_thread () from /lib64/libpthread.so.0
#7  0x00007ffff6ecc8cd in clone () from /lib64/libc.so.6

@bnoordhuis - thanks! the first assertion has this backtrace; not able to obtain for the second one. Looks like it is always triggered by a worker thread, while the main thread is already processing the first assertion failure, because the second one occurs only when first one is present (this is my guess, no proof)

running out of file descriptors looks like a possibility; I will debug from that angle.

gireeshpunathil

comment created time in 20 days

issue commentnodejs/node

assertion failure in src/node_worker.cc with large number of workers

/cc @nodejs/libuv

gireeshpunathil

comment created time in 20 days

issue commentnodejs/node

assertion failure in src/node_worker.cc with large number of workers

node: ../deps/uv/src/unix/core.c:556: uv__close_nocheckstdio: Assertion `fd > -1' failed.

In the assertion failure list, I see this too.

gireeshpunathil

comment created time in 20 days

issue openednodejs/node

assertion failure in src/node_worker.cc with large number of workers

<!-- Thank you for reporting a possible bug in Node.js.

Please fill in as much of the template below as you can.

Version: output of node -v Platform: output of uname -a (UNIX), or version and 32 or 64-bit (Windows)

Subsystem: if known, please specify the affected core module name

If possible, please provide code that demonstrates the problem, keeping it as simple and free of external dependencies as you can. -->

  • Version: master, v14.0.0-pre
  • Platform: Linux 3.10.0-957.5.1.el7.x86_64 1 SMP Wed Dec 19 10:46:58 EST 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Subsystem: workers

<!-- Please provide more details below this comment. -->

I was debugging https://github.com/nodejs/node/issues/23277 and came across this:

$ cat bar.js

const { Worker } = require('worker_threads');

for (let i = 0; i < 10000; ++i) {
  const worker = new Worker(
    'require(\'worker_threads\').parentPort.postMessage(2 + 2)',
    { eval: true });
}

$ node --max-old-space-size=100000 bar

node[101402]: ../src/node_worker.cc:135:node::worker::WorkerThreadData::WorkerThreadData(node::worker::Worker*): Assertion `(uv_loop_init(&loop_)) == (0)' failed.

I guess this has to do with libuv failure due to lack of memory, but can this be better handled?

/cc @nodejs/workers

created time in 20 days

issue commentkubernetes/kubernetes

Automatic PV(C) support

@hprateek43 - thanks for the response!

my use cases are:

  • dumps - (system dumps, snap dumps, heap dumps): they aid diagnostics, and need to persist across restarts
  • traces and logs - according to discretion, an architect want to design the runtime / middleware / application specific tracing and logging to be written to disk.

while this is possible through configuration on a per deployment level, both these requirements are scoped wide, at the project level on a minimum.

And hence it makes sense for a project user to be able to configure a PV and a PVC that applies across the board?

gireeshpunathil

comment created time in 21 days

issue commentnodejs/node

Investigate flaky parallel/test-worker-memory

I am now tending to suspect that the behavior explained in https://github.com/nodejs/node/issues/21266#issuecomment-580655540 apply here. Therss memory to quiesce fully and come back to near-normal, there should be memory demand in other parts of the system, or else it just stays active in the current process.

Trott

comment created time in 21 days

issue commentnodejs/help

Clarify example for AsyncResource

@daprahamian - pls review https://github.com/nodejs/node/pull/31601 and see if that helps you

daprahamian

comment created time in 21 days

issue commentkubernetes/kubernetes

Automatic PV(C) support

/sig storage /sig usability /sig cluster-lifecycle

gireeshpunathil

comment created time in 21 days

issue openedkubernetes/kubernetes

Automatic PV(C) support

<!-- Please only use this template for submitting enhancement requests -->

What would you like to be added:

  • automatic provision of a pvc, one per POD
  • configurable at the project level, with default being disabled.
  • applied for every deployments under the project
  • size of the volume as a function of the size of the container that runs the app / service
  • retain last n volumes, with automatic cleanup of the rest, with n defaults to 1 and is configurable

Why is this needed:

By default, POD crashes lead to recycle without leaving a trace. This design works well for high availability philosophy, but acts against serviceability principles. Traditional (e.g:- con-containerized) deployments did not suffer from this issue.

Production deployments always need to have a way to persist some data - traces, dumps, logs etc. for some period of time that is mostly more than the life time of the container; so the day 2 operators always configure a PV for every deployment. If the platform supports a way to achieve this through standard semantics, that will improve the diagnostic story greatly.

created time in 21 days

issue commentappsody/stacks

Nodejs-Express stack build gives error `error building at STEP "RUN yum upgrade`

@sam-github - It was my suggestion, based on a glance at the first failure data. From the error log, it looks like we are executing https://github.com/kabanero-io/draft-guide-dev-getting-started/blob/c7dff651363e699627c002a0610a70f1b9b95923/collections/incubator/nodejs-express/image/project/Dockerfile

@aadeshpa - can you confirm?

Trying to understand why would you need to init the project on the host ?

the behavioral differences are because of the difference in the host - as the container used are the same.

aadeshpa

comment created time in 22 days

pull request commentexpressjs/session

doc: add info about DEBUG mode

I actually perceive it to be the exact opposite,

ok, I may be wrong, and I don't have statistics. I bank on your perceptions as the lead maintainer of this project. Just reversed the example. PTAL!

gireeshpunathil

comment created time in 22 days

push eventgireeshpunathil/session

Gireesh Punathil

commit sha be610cb6f1b95d311b3fcdd921cf26cca6b78f4c

fixup: reverse examples between win and unix

view details

push time in 22 days

pull request commentexpressjs/session

doc: add info about DEBUG mode

Maybe you can help me understand why should there be an example for the syntax of export but not for the syntax of set?

It is based on the perceived user base: UNIX used more than Windows.

gireeshpunathil

comment created time in 22 days

pull request commentexpressjs/session

doc: add info about DEBUG mode

Right, and that's why I suggest we just copy what is on expressjs.com page I shared, as that seems to have calmed down the PRs.

@dougwilson - I have to disagree with your reasoning here. presence of a piece of text / code / doc did not invite further PR is not a good reason to infer that that is the text which is better for consumption. Potential reasons for less PRs could be, but not limited to:

  • seen less
  • less needed to debug

This is a documentation change. IMO its merit is best measured in terms of

  • correctness
  • usability
  • preciseness

It is sad to see you block this PR based on a reason that is outside of these!

gireeshpunathil

comment created time in 22 days

Pull request review commentexpressjs/session

doc: add info about DEBUG mode

 This is a [Node.js](https://nodejs.org/en/) module available through the $ npm install express-session ``` +## Debugging++```sh+$ export DEBUG=express-session

ok , I understand your reasons. Such users (who cannot reasonably follow the document) can also copy-paste the dollar symbol, and app.js from the phrase $ DEBUG=express-session node app.js while there shell prompt and app name might have been something else?

my take would be to balance between readability and usability, while accepting reasonable changes (in the future) and provide explanation to the rest.

gireeshpunathil

comment created time in 22 days

Pull request review commentexpressjs/session

doc: add info about DEBUG mode

 This is a [Node.js](https://nodejs.org/en/) module available through the $ npm install express-session ``` +## Debugging++```sh+$ export DEBUG=express-session

may be we all 3 talked almost at the same time, so I am not sure about the ordering of the above conversation - who is asking, and who is answering to whom! :)

gireeshpunathil

comment created time in 22 days

Pull request review commentexpressjs/session

doc: add info about DEBUG mode

 This is a [Node.js](https://nodejs.org/en/) module available through the $ npm install express-session ``` +## Debugging++```sh+$ export DEBUG=express-session

Setting this property prior to the run (use 'set' in Windows) - the first line in my change has this, that should clarify, and work I guess?

gireeshpunathil

comment created time in 22 days

Pull request review commentexpressjs/session

doc: add info about DEBUG mode

 This is a [Node.js](https://nodejs.org/en/) module available through the $ npm install express-session ``` +## Debugging++```sh+$ export DEBUG=express-session

@dougwilson - my motivation is to document critical, yet missing information that aids self-diagnosing issues of using this module. Reading it multiple times, I don't see any misleading, confusing or redundant phrases, nor any incomplete info that will / may invite future bike-shedding.

I would like to retain the changes. I see you did not use the red button, so I treat your comment as a suggestion that is free to leave out, not a blocker.

gireeshpunathil

comment created time in 22 days

Pull request review commentexpressjs/session

doc: add info about DEBUG mode

 This is a [Node.js](https://nodejs.org/en/) module available through the $ npm install express-session ``` +## Debugging++```sh+$ export DEBUG=express-session+```++Setting this property prior to the run (use `set` in Windows)+enables `debug` mode, and prints minimal but vital debug

thanks, fixed it.

gireeshpunathil

comment created time in 22 days

push eventgireeshpunathil/session

Gireesh Punathil

commit sha 3233c00855af369e57191d719da52b7390280538

fixup: address review comments

view details

push time in 22 days

issue commentappsody/stacks

Nodejs-Express stack build gives error `error building at STEP "RUN yum upgrade`

what happens if you do appsody init nodejs-express directly on the host?

aadeshpa

comment created time in 22 days

issue commentkabanero-io/kabanero-pipelines

Bug: performance problem in build-bud step of nodejs and nodejs loopback pipeline

I will debug item (2) of the above.

tam512

comment created time in 23 days

Pull request review commentnodejs/node

process: report ArrayBuffer memory in `memoryUsage()`

 Will generate:   rss: 4935680,   heapTotal: 1826816,   heapUsed: 650472,-  external: 49879+  external: 49879,+  arrayBuffers: 9386 } ``` -`heapTotal` and `heapUsed` refer to V8's memory usage.-`external` refers to the memory usage of C++ objects bound to JavaScript-objects managed by V8. `rss`, Resident Set Size, is the amount of space-occupied in the main memory device (that is a subset of the total allocated-memory) for the process, which includes the _heap_, _code segment_ and _stack_.--The _heap_ is where objects, strings, and closures are stored. Variables are-stored in the _stack_ and the actual JavaScript code resides in the-_code segment_.+* `heapTotal` and `heapUsed` refer to V8's memory usage.+* `external` refers to the memory usage of C++ objects bound to JavaScript+  objects managed by V8.+* `rss`, Resident Set Size, is the amount of space occupied in the main+  memory device (that is a subset of the total allocated memory) for the+  process, including all C++ and JavaScript objects and code.+* `arrayBuffers` refers to memory allocated for `ArrayBuffer`s and+  `SharedArrayBuffer`s, including all Node.js [`Buffer`][]s.+  This is also included in the `external` value. When Node.js is used as an+  embedded library, this value may be `0`.

there is nothing probabilistic involved here, I think?

I termed it as probabilistic because of may be 0 phrase. But based on your explanation to the rest of the questions, I understand it as:

if embedders use only non-node.js allocators without a tracker, then this value WILL be 0 (not truly reflecting actual consumption), and if embedders rely on node.js' allocators, then the value will be accurate - zero or non-zero

Are we on the same page?

addaleax

comment created time in 23 days

issue commentnodejs/node

Diagnostic report on fatal error does not respect settings

https://github.com/nodejs/node/pull/29881 is indeed an attempt to address this. Pinging @shobhitchittora to see where do we stand on this

brandondoran

comment created time in 24 days

Pull request review commentnodejs/node

process: report ArrayBuffer memory in `memoryUsage()`

 Will generate:   rss: 4935680,   heapTotal: 1826816,   heapUsed: 650472,-  external: 49879+  external: 49879,+  arrayBuffers: 9386 } ``` -`heapTotal` and `heapUsed` refer to V8's memory usage.-`external` refers to the memory usage of C++ objects bound to JavaScript-objects managed by V8. `rss`, Resident Set Size, is the amount of space-occupied in the main memory device (that is a subset of the total allocated-memory) for the process, which includes the _heap_, _code segment_ and _stack_.--The _heap_ is where objects, strings, and closures are stored. Variables are-stored in the _stack_ and the actual JavaScript code resides in the-_code segment_.+* `heapTotal` and `heapUsed` refer to V8's memory usage.+* `external` refers to the memory usage of C++ objects bound to JavaScript+  objects managed by V8.+* `rss`, Resident Set Size, is the amount of space occupied in the main+  memory device (that is a subset of the total allocated memory) for the+  process, including all C++ and JavaScript objects and code.+* `arrayBuffers` refers to memory allocated for `ArrayBuffer`s and+  `SharedArrayBuffer`s, including all Node.js [`Buffer`][]s.+  This is also included in the `external` value. When Node.js is used as an+  embedded library, this value may be `0`.

sorry for being raw here, but I have these questions:

When Node.js is used as an embedded library, this value may be 0.

  • why? is it because Node.js cannot truly account for the allocations of the embedder?
  • where? which part of the code takes the decision based on the embedded state?
  • may be 0 - why again - why this decision cannot be deterministic as opposed to probabilistic? which factors decide the outcome? how would one use those factors to decide whether the value is trustworthy or not?
addaleax

comment created time in 24 days

issue commentnodejs/TSC

Node.js Foundation Technical Steering Committee (TSC) Meeting 2020-01-29

unforeseen personal event; I will miss this.

mhdawson

comment created time in 24 days

issue commentkabanero-io/kabanero-pipelines

Bug: performance problem in build-bud step of nodejs and nodejs loopback pipeline

@aadeshpa - please see comments https://github.com/kabanero-io/kabanero-pipelines/issues/121#issuecomment-578009040 and https://github.com/kabanero-io/kabanero-pipelines/issues/121#issuecomment-579044448 . To resolve this issue quickly, we need to isolate platforms where it works and where it does not, out of the box or with tweaking, and based on count, the versions where it works etc. we may take a call about whether this approach (fuse bug fix and buildah bug fix) is acceptable or not.

Pipeline testing can be complex; and as you and @jdmcclur also have pointed out, there could be unrelated issues hiding somewhere else (OS upgrade issue / Dockerfile syntax issue etc). We want to isolate all of those and diagnose and resolve the performance + mount bug through this issue. For that, I recommend we follow https://github.com/kabanero-io/kabanero-pipelines/issues/121#issuecomment-578009040

tam512

comment created time in 25 days

issue commentnodejs/diagnostics

Node.js Foundation Diagnostics WorkGroup Meeting 2020-01-29

We will be discussing https://github.com/nodejs/diagnostics/issues/349 in this sitting. Inviting relevant stake holders and interested parties

  • @addaleax
  • @jasnell
  • @cjihrig
  • @boneskull
  • @june07
  • @richardlau
mhdawson

comment created time in 25 days

issue commentkabanero-io/kabanero-pipelines

Bug: performance problem in build-bud step of nodejs and nodejs loopback pipeline

@aadeshpa - I am no OS upgrade specialist, but I recommend to drop that route and follow my recommendation in the previous comment, which is much easier to run and observe result. Once we have a set, you could setup cluster in those representative systems and test with minimal effort

tam512

comment created time in a month

pull request commentnodejs/node

doc: guide - using valgrind to debug memory leaks

@gireeshpunathil, @pmarton FYI

reasonably sure you meant to ping @hekike

mhdawson

comment created time in a month

Pull request review commentnodejs/node

doc: guide - using valgrind to debug memory leaks

+# Investigating Memory Leaks with valgrind++A Node.js process may run out of memory due to excessive consumption of+`Native memory`. `Native Memory` is memory which is not managed by the the+V8 Garbage collector and is allocated either by the Node.js runtime, it's+dependancies or native [addons](://nodejs.org/docs/latest/api/n-api.html).++This guide provides informaiton on how to use valgrind to investigate these+issues.++## valgrind++[Valgrind](https://valgrind.org/docs/manual/quick-start.html) is a+tool available on linux distributions which can be used to investigate+memory usage including identifying memory leaks (memory which is+allocated and not freed) and other memory related problems+like double freeing memory.++To use valgrind:++* Be patient, running under valgrind slows execution significantly due+  to the checks being performed.+* Reduce your test case to the smallest reproduce. Due to the slowdown it is+  important to run the minimum testcase in order to be able to do it in+  a reasonable time.

Reduce your test case to the smallest reproduce

also important as it reduces the complexity and reduces the overall amount of debug log to analyse?

mhdawson

comment created time in a month

pull request commenttektoncd/experimental

plgen: tekton DSL based on Dockerfile

@vdemeester - thanks for the review!

Dockerfile is really limited in what it can do : there is no graph involved, and there is some command in it (like USER) that are weirdly translatable in a Task.

  • my premise is that pipeline tasks are imperative in nature, that fits the Dockerfile philosphy
  • I kind of agree, the existing Dockerfile verbs are limited, but plgen parser is extensible to fit pipeline needs
  • it allows to define and standardize new verbs that are most natural for the pipeline tasks.
  • pipeline tasks themselves are not too complex, and can be seen as well scoped in a minimal set of vocabulary

Dockerfile are meant to generate an oci-image in the end (and most of the time are written this way) and this assumption is very limited to the translation into Task (your experiment doesn't seem to generate images).

  • of course, we are not using the syntax to create images, instead for defining pipelines
  • Dockerfile is an analogy, meant for fast ramp up and easy representation
  • the analogy should become evident to any consumer on the first use
  • software programming domain is filled with such analogies: print, thread, kill, gc, throw, native ... no user will associate these keywords with their original meaning, beyond the first use IMO.

There is a few weird thing in the generated yamls, that I don't think work

  • agreed, I can get back to my branch, familiarize with the code (it has been some time) and fix those; but given that there are design level questions that needs to be ratified; I will delay the bug fixing until then.

In general, I appeal to contemplate this approach as independent from yaml semantics as possible - In my experience, the problem of comparing with an established baseline is that:

  • the gaps of comparing entity from the baseline gets highlighted more
  • any novelty in the comparing entity gets masked off, as those don't appear for comparison, because those are missing in the baseline.
gireeshpunathil

comment created time in a month

issue commentnodejs/diagnostics

reportVersion semantics are not defined

thanks @june07 for the insights! Feel free to join our one of the upcoming diagnostic WG meetings when we take this up. the next one is going to be on 29th, but unlikely to pick this one up (due to pre-decided agenda, pinging @hekike for a confirmation. )

addaleax

comment created time in a month

pull request commenttektoncd/experimental

plgen: tekton DSL based on Dockerfile

nearing 3 months with no review - not a great way to motivate new contributors! let me know what is the blocker here:

  • too much code to review? I can present it in one of the forums you suggest
  • lacking a description / flow? I have attempted to sketch one, but pls suggest if needs more
  • lacking review time? I am happy to wait, but pls state so!
gireeshpunathil

comment created time in a month

issue commentnodejs/help

ETIMEDOUT error most of the time

the conn represent the the connection between salesforce and node.js

ok, but it may be an implementation in your program, or an npm module, not an node.js API (because there is no sobject function in any of our HTTP/S implementations). that is why I suggested to identify and check with the module.

mayankagarwal900

comment created time in a month

issue commentkabanero-io/kabanero-pipelines

Bug: performance problem in build-bud step of nodejs and nodejs loopback pipeline

ok, given

  • the impact of this issue,
  • and a bit of confusion caused due to many issues reported upstream that may or may not be related leading to unertainty about which package / OS versions contain the fix,

I suggest the following:

  • we test our use case on a wide variety of platforms [ rhel / centos / ubuntu / any other?] where fuse bug apply
  • for ease of testing, use the simple recreate, listed in the first comment of https://github.com/containers/buildah/issues/2047
  • figure out which one works out of the box, and which one works with package (fuse-verlay) upgrade
  • confirm correctness with full-blown pipeline testing in representative platforms, with lowest working versions
  • document the platform requirement, in https://github.com/kabanero-io/kabanero-pipelines 's readme

if people agree, here is a matrix. Feel free to edit or add your entries if you have tested on that combination.

OS with vanilla os with fuse upgrade person tested
centos 7.6
centos 8.0
rhel 7.6 BAD TODO gireesh
rhel 7.7
rhel 7.8
rhel 8.0 GOOD NA gireesh
rhel 8.1
ubnt 16.04
ubnt 18.04
tam512

comment created time in a month

issue commentnodejs/help

ETIMEDOUT error most of the time

ok, thnx. what that object conn represents? you might need to check the doc for sobject API in that module's documentation, to see what is the default timeout, and how to override that.

mayankagarwal900

comment created time in a month

issue commentnodejs/help

ETIMEDOUT error most of the time

file being larger should cause timeout for a well designed API IMO. As long as the flow is ON, it should treat the connection as active. The timeout should kick-in only for inactive connections, counting from the time the last byte sent.

I may be wrong.

mayankagarwal900

comment created time in a month

issue commentnodejs/help

ETIMEDOUT error most of the time

that depends on:

  • which API you are using to make the request
  • and whether you are relying on the default timeout defined by the API, or overriding it

can you share your that piece of code?

mayankagarwal900

comment created time in a month

issue commentnodejs/help

ETIMEDOUT error most of the time

@mayankagarwal900 - ok, so the connection was previously established, and then it times out in the middle of it?

if so I would not debug at the node side, instead at the other end point.

In either way, employing tcpdump or wireshark to see what happens in the wire seems to be a good idea for me.

mayankagarwal900

comment created time in a month

issue commentnodejs/help

ETIMEDOUT error most of the time

@mayankagarwal900 - from the box where you host the server, can you try pinging the target system and see what happens?

mayankagarwal900

comment created time in a month

issue commentnodejs/help

How to change the location `.config` and `.node_repl_history` in Windows 7

@Templore -

#cd /tmp
#export NODE_REPL_HISTORY=./foo.txt
#node
Welcome to Node.js v13.0.1.
Type ".help" for more information.
> process.version
'v13.0.1'
> 
(To exit, press ^C again or ^D or type .exit)
> 
#cat ./foo.txt 
process.version#
#ls ~/.config/
browser-launcher2/ configstore/       yarn/              

#npm config --help
npm config set <key> <value>
npm config get [<key>]
npm config delete <key>
npm config list [--json]
npm config edit
npm set <key> <value>
npm get [<key>]

#npm config ls -l | grep userconfig
userconfig = "/Users/gireeshpunathil/.npmrc"

#npm config set userconfig /tmp/.npmrc
#npm config ls -l | grep userconfig
userconfig = "/tmp/.npmrc"

#

hope this helps!

Templore

comment created time in a month

issue commentnodejs/node

fs.readFile possible memory leak

also, is there a race condition by which multiple threads believe (together) that they need to expand?

ip413

comment created time in a month

issue commentnodejs/node

fs.readFile possible memory leak

expanded v8 heaps are never shrunk, even if the usage is long ceased?

ip413

comment created time in a month

issue commentnodejs/node

fs.readFile possible memory leak

but this one looks like heap expansion!

ip413

comment created time in a month

issue commentnodejs/node

fs.readFile possible memory leak

(gdb) b mmap if $rsi >= 134217728
Starting program: node --expose-gc 21266
...
Breakpoint 2, 0x00007ffff6ec6d50 in mmap64 () from /lib64/libc.so.6
(gdb) where
#0  0x00007ffff6ec6d50 in mmap64 () from /lib64/libc.so.6
#1  0x00007ffff6e4d231 in new_heap () from /lib64/libc.so.6
#2  0x00007ffff6e4dc64 in arena_get2.isra.3 () from /lib64/libc.so.6
#3  0x00007ffff6e53b0e in malloc () from /lib64/libc.so.6
#4  0x00007ffff792eecd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#5  0x0000000000d87c68 in std::__detail::_Map_base<v8::internal::MemoryChunk*, std::pair<v8::internal::MemoryChunk* const, v8::internal::MemoryChunkData>, std::allocator<std::pair<v8::internal::MemoryChunk* const, v8::internal::MemoryChunkData> >, std::__detail::_Select1st, std::equal_to<v8::internal::MemoryChunk*>, v8::internal::MemoryChunk::Hasher, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](v8::internal::MemoryChunk* const&) ()
#6  0x0000000000d88681 in v8::internal::ConcurrentMarkingVisitor::ShouldVisit(v8::internal::HeapObject)
    ()
#7  0x0000000000d8e4f4 in v8::internal::ConcurrentMarking::Run(int, v8::internal::ConcurrentMarking::TaskState*) ()
#8  0x0000000000d0b283 in v8::internal::CancelableTask::Run() ()
#9  0x0000000000b05dc9 in node::(anonymous namespace)::PlatformWorkerThread(void*) ()
#10 0x00007ffff71a3ea5 in start_thread () from /lib64/libpthread.so.0
#11 0x00007ffff6ecc8cd in clone () from /lib64/libc.so.6
(gdb) i r $rsi
rsi            0x8000000	134217728
(gdb) 
ip413

comment created time in a month

issue commentnodejs/node

fs.readFile possible memory leak

I am able to see the issue.

0000000004a34000 67944K rw--- [ anon ]

sections such as this, never gets unmapped.

strace also shows a similar story:

[pid 38519] 1579773078.495439 mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0 <unfinished ...>
[pid 38520] 1579773078.495481 mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0 <unfinished ...>
[pid 38519] 1579773078.495489 <... mmap resumed> ) = 0x7f4640000000
[pid 38520] 1579773078.495516 <... mmap resumed> ) = 0x7f4638000000

there is no matching munmap calls.

ip413

comment created time in a month

issue commentnodejs/diagnostics

Improving automated remote connection via Inspector Protcol

  • +1 to everything what was said.
  • in terms of capability, telling inspector to inhibit the client from modifying program state will do?
mmarchini

comment created time in a month

issue commentnodejs/diagnostics

reportVersion semantics are not defined

I agree with @boneskull . Given that report is a diagnostic feature, IMO modification is all about more data, less data or structural changes, and the main party affected is tools. semver means adding more complexity here.

pinging @june07 (who developed an inspector plugin that understands and renders report https://github.com/june07/NiM ) for an opinion

addaleax

comment created time in a month

PR closed nodejs/node

Just adding a comment build doc

<!-- Thank you for your pull request. Please provide a description above and review the requirements below.

Bug fixes and new features should include tests and possibly benchmarks.

Contributors guide: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md -->

Checklist

<!-- Remove items that do not apply. For completed items, change [ ] to [x]. -->

  • [ ] make -j4 test (UNIX), or vcbuild test (Windows) passes
  • [ ] tests and/or benchmarks are included
  • [ ] documentation is changed or added
  • [ ] commit message follows commit guidelines

<!-- Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

(b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

(c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

(d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. -->

+1 -1

0 comment

1 changed file

ianshalom

pr closed time in a month

issue commentkabanero-io/kabanero-pipelines

Bug: performance problem in build-bud step of nodejs and nodejs loopback pipeline

@aadeshpa - thanks for the tests.

is this the fuse overlay issue you were saying

can't say for sure. can you please:

  • share the host details? cat /etc/os-release
  • share the steps to reproduce?
tam512

comment created time in a month

issue commentnodejs/help

Connecting to MongoDB

@edgarascom -

I use .evn to set up the DATABASE_URL:

can you explain what is .evn ? and what is special with that? did you mean environment variable exported / set on the invoking shell?

if so, here is how you would do it:

$ export DATABASE_URL=mongodb://localhost/nodejs-app-db

and then, use

process.env.DATABASE_URL

in the code. Please note env as opposed to evn

edgarascom

comment created time in a month

issue closednodejs/help

On creation of new product the Dismiss button need to clear the values entered

Is your feature request related to a problem? Please describe. The Dismiss button should clear the values for any product but when i start provisining the same product i see the values left in it.

Describe the solution you'd like On click of Dismiss button the fields should be cleared.

Steps Select Products, start provisining a VM/AEM/Openshift etc.. Click on Dismiss Open the same product and try to provision the same product - you would see the values entered previously.

closed time in a month

naregal4

issue commentnodejs/help

On creation of new product the Dismiss button need to clear the values entered

@naregal4 - how this problem statement is related to node.js? you most probably reported this in a wrong repo. can you check?

naregal4

comment created time in a month

issue closednodejs/help

Add text when no billable ASK ID found

Is your feature request related to a problem? Please describe. The search ASK ID component will only show ASK IDs that have GL_ASSIGNED = Yes. This is to make sure that projects, VMs, etc are not craeted against a non-billable ASK ID. If a user searches for an ASK that exists, but is not billable, the form just returns 0 results.

Describe the solution you'd like We have had feedback from NPS, users and incidents that users are confused and ask questions about why it is not showing up. It would be good if we could display red text under the field that says something like "No Billable ASK IDs found" or something that would inform the user that they one they searched for can't be used.

OR

Show all ASK names in the list and when someone clicks on an ASK name that hast GL_ASSIGNED=NO, then display text saying it's not billable and can't be used (and maybe link to the instructions on how to assign a GL

closed time in a month

naregal4

issue commentnodejs/help

Add text when no billable ASK ID found

@naregal4 - how this problem statement is related to node.js? you most probably reported this in a wrong repo. can you check?

naregal4

comment created time in a month

issue commentnodejs/node

Silent JSON.parse() crash when reading multiple large files very fast.

@QuentinFAIDIDE - did you get confirmation from the OS logs about the details / reason for the crash?

QuentinFAIDIDE

comment created time in a month

more