profile
viewpoint

animir/node-rate-limiter-flexible 1994

Node.js rate limit requests by key with atomic increments in single process or distributed environment.

animir/nodebestpractices 3

The largest Node.JS best practices list (November 2018)

animir/awesome-nodejs 1

:zap: Delightful Node.js packages and resources

animir/alltables 0

parse tables for alltables.info

animir/angular-clock 0

angular clock widget directive: Demo-

animir/awesome-nodejs-security 0

Awesome Node.js Security resources

animir/bull 0

Premium Queue package for handling distributed jobs and messages in NodeJS.

animir/choise 0

Generate functions

animir/codility-go 0

Solutions for the codility.com exercises in Go

animir/connect-ensure-login 0

Login session ensuring middleware for Connect and Express.

startedhyvyys/Tektur

started time in 9 days

issue commentanimir/node-rate-limiter-flexible

Using rate-limit-flexible with cloudflare workers

@LGmatrix13 Hey. Which type of limiter from rate-limiter-flexible do you use? I presume, you use RateLimiterMemory?

I think, it is possible to implement a new type of limiter RateLimiterCloudFlare, since there is this counter example https://developers.cloudflare.com/workers/learning/using-durable-objects#example---counter. However, performance of this approach could be low because of additional concurrency block during increment operation, as this is the only way increment can be atomic on CloudFlare. Also, there is no TTL, so it would require TTL as separate value stored with Durable Objects. Worth trying, if you are not in harry.

Regarding storing in memory between requests, here is a quote from the https://developers.cloudflare.com/workers/platform/limits#memory nodes are occasionally evicted from memory, that means it can not be used across all node(s).

LGmatrix13

comment created time in 12 days

push eventanimir/node-rate-limiter-flexible

Roman Voloboev

commit sha 3cdb40bd4cbf78eb551c7d68dcf89f25934cf9a3

add .github to npm ignore

view details

push time in 15 days

push eventanimir/node-rate-limiter-flexible

Roman

commit sha 306ef3311bdd8a56b5d70658802ceaaeb8057e08

Create FUNDING.yml

view details

push time in 15 days

issue closedanimir/node-rate-limiter-flexible

Throttlers and rate limiters, how they should be implemented

Hello @animir , Thank you for your work on this package.

I'm raising this to discuss the current implementation,

voila

Click for dark mode users

closed time in 17 days

bacloud14

issue commentanimir/node-rate-limiter-flexible

Throttlers and rate limiters, how they should be implemented

@bacloud14 Hey, thanks for the topic raised.

FIrst of all, with rate-limiter-flexible any event can be counted, not necessary request from a client. Every event may have its own priority and consume more/less points to make the whole system balanced.

Secondly, you can set the maximum time of a process as points option for a limiter and consume milliseconds spent by some unique process key from a limiter. It would be not easy to implement process interruption, though. If it is necessary to interrupt a process, you probably need your own limiter implementation or another package.

Btw, how would you know, what is maximum time for a process?

PS: thanks for the dark mode link :)

bacloud14

comment created time in 17 days

issue commentanimir/node-rate-limiter-flexible

RateLimiterRedis don't aplly params to define limiters

@brocchirodrigo We faced some issues with redis v4.x before. Please, use previous version of redis package. If you fix RateLimiterFlexible to work with redis version 4, please, create a PR with changes.

brocchirodrigo

comment created time in 19 days

GollumEvent

issue commentanimir/node-rate-limiter-flexible

RateLimiterRedis don't aplly params to define limiters

@brocchirodrigo What redis package version do you use?

brocchirodrigo

comment created time in 19 days

issue commentanimir/node-rate-limiter-flexible

RateLimiterMemory: clearExpiredByTimeout

@tomjez @LGmatrix13 Hi, RateLimiterMemory doesn't have the clearExpiredByTimeout option as it is in the nature of this limiter. RateLimiterMemory sets timeouts to delete a key depending on your limiter's options.

Could you provide more details on how many keys you have and how fast memory footprint grows?

tomjez

comment created time in 20 days

issue commentanimir/node-rate-limiter-flexible

RateLimiterRedis don't aplly params to define limiters

@brocchirodrigo Hey, what redis package and version do you use? Could you share a link with README you're talking about?

brocchirodrigo

comment created time in 21 days

issue closedanimir/node-rate-limiter-flexible

Combining multiple rate limiters atomically

I want to implement an API rate limiter using Redis. On receiving a request, I call rate limiter.consume(), and then await ratelimiter.get() to check if limits have crossed or not. While using this in a distributed system, is it possible 2 commands get executed simultaneously for consume, and even though the counters are incremented atomically for both, both threads get the updated value of the counter? for example, if the limit is 100, and the current count is 99, and 2 requests come in, where both the consume functions are executed simultaneously, and the counter becomes 101. now both get functions receive 101 as counter value, and therefore reject both the requests, but 1 of them should have been allowed. Am I missing something?

Also, I wanted to create rate limiter having limits for per sec, per minute, per hour, and per day. Currently, I implemented this by creating 4 instances of RateLimiterRedis with different duration and prefix parameters. when a request come in, I call .consume() on all the 4 instances and then call .get() to get the current status and check if rate limits have crossed or not. But in this I am basically making 4 consume calls and 4 get calls, which is taking the whole time of ~250 ms. Can this be somehow reduced by doing the increment of 4 counters in a single call to redis using redis scripts?

closed time in 25 days

akashround

issue commentanimir/node-rate-limiter-flexible

Combining multiple rate limiters atomically

@akashround Hey, I hope you solved this issue. I am closing this, feel free to re-open if something pops up.

akashround

comment created time in 25 days

issue closedanimir/node-rate-limiter-flexible

Typescript implementation example

Hi, I'm trying to create a new class that extends the RateLimiterStoreAbstract, but after extending it I got the below error. What I am doing wrong? Could you give me a typescript sample to implement it or provide a RateLimiterS3?

Class extends value undefined is not a constructor or null

closed time in a month

thiagoolsilva

issue commentanimir/node-rate-limiter-flexible

Typescript implementation example

@thiagoolsilva Hey, I am closing this, feel free to re-open, if you are going to work on this issue or you have any questions.

thiagoolsilva

comment created time in a month

fork animir/API-template

template for Voiceflow take-home project

fork in a month

issue commentanimir/node-rate-limiter-flexible

Combining multiple rate limiters atomically

@akashround

could this be a reason?

I don't think, rate-limiter-flexible tested with async-redis package. most likely, it doesn' work because of async-redis.

there seems to be some breaking changes with the redis 4.0 release

Yes, there an open issue for that https://github.com/animir/node-rate-limiter-flexible/issues/133 You can use ioredis instead of redis package, until rate-limiter-flexible is improved to work with redis v4.

Will be great if you can update the docs regarding this

Thanks, I'll update docs, when we fix compatibility with node-redis v4 package.

akashround

comment created time in a month

issue commentanimir/node-rate-limiter-flexible

Combining multiple rate limiters atomically

Okay, sounds good. I will try that. Any idea about why it is blocking when using async/await and why I am not able to access rateLimterRes object inside .then() ?

Did you cover await call by try/catch block? You should be able to access rateLimterRes in catch block. Not sure about then, if promise is resolved, there should be some value anyway. Could you double-check this, before we dig dipper?

akashround

comment created time in a month

issue commentanimir/node-rate-limiter-flexible

Combining multiple rate limiters atomically

@akashround

I am not including anything for points

Points option is still set by default. You can set points to some huge value and check consumedPoints from rateLimiterRes instead of using get method. In this case a request would never be rate limited until you decide it looking at consumedPoints for every client separately.

I hope, this helps.

akashround

comment created time in a month

issue commentanimir/node-rate-limiter-flexible

Combining multiple rate limiters atomically

@akashround Hey, first of all, it is not necessary to call get after consume. Consume method returns an object of RateLimiterRes class. Its description can be found in the readme of this package.

Redis script implemented for rate-limiter-flexible package doesn't support more than one increment per request. You could fork master branch and create your own implementation, if it is crucial.

Btw, how do you make 4 consme calls? Do you make them all async? 250ms is too much for 4 async consume requests and then 4 async get requests.

akashround

comment created time in a month

startedanimir/node-rate-limiter-flexible

started time in a month

issue commentanimir/node-rate-limiter-flexible

Typescript implementation example

@thiagoolsilva Hi, typescript definitions may be not complete enough.

You found the list of required methods. Do you have an idea how would you implement those for S3?

thiagoolsilva

comment created time in a month

issue commentanimir/node-rate-limiter-flexible

Typescript implementation example

@thiagoolsilva This is interesting idea. How would you implement atomic upsert for incrementing counter there on S3?

Could you provide the code you already developed to get that error message?

thiagoolsilva

comment created time in a month

issue commentanimir/node-rate-limiter-flexible

Typescript implementation example

@thiagoolsilva Hi, could you provide your code?

Also, could you describe, what would you like to achieve with the new RateLimiterS3 class?

thiagoolsilva

comment created time in a month

GollumEvent

PR closed animir/node-rate-limiter-flexible

added install size badge

hello! this PR adds an 'install size' badge that can be seen here https://packagephobia.com/badge?p=rate-limiter-flexible

rate-limiter-flexible is a terrific package that is amazingly small as well and I hope you will accept this PR as a token of gratitude for this excellent package.

+3 -0

3 comments

1 changed file

iambumblehead

pr closed time in a month

pull request commentanimir/node-rate-limiter-flexible

added install size badge

@iambumblehead good idea on build:passing badge, thanks.

iambumblehead

comment created time in a month

push eventanimir/node-rate-limiter-flexible

Roman Voloboev

commit sha e63f3b2c954211399e10cd30516ef04c12485a68

remove build badge from readme

view details

push time in a month

pull request commentanimir/node-rate-limiter-flexible

added install size badge

@iambumblehead Thank you for this PR. I do have some doubts it is useful. First of all, there are already a lot of images and text in readme. It'd be better to add less info if possible. Secondly, npm displays package size, so for somebody who cares about package size, it is easy to check it there. What do you think?

iambumblehead

comment created time in a month

startedanimir/node-rate-limiter-flexible

started time in a month

more