profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/wokalski/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.
Wojtek Czekalski wokalski Poland wokalski.com tweet @wokalski

revery-ui/revery 7717

:zap: Native, high-performance, cross-platform desktop apps - built with Reason!

rescript-react-native/rescript-react-native 837

ReScript bindings for React Native

nars-dev/nars 85

Server rendered React Native

wokalski/autocomplete-flow 76

Neovim and vim Flow autocompletion for deoplete + neosnippet

szymonzmyslony/reason-react-native-navigation 27

This is a simple react native navigation written in reason.

esy/github-action 15

Github action for esy

saschatimme/reason-ui-explorer 15

Port of the React Native UIExplorer to Reason

esy-ocaml/esy-opam 5

opam to esy package converter

pull request commentgraphile/worker

Add support for arbitrary functions in place of the cron pattern

Makes sense. I will try to do a couple of small PRs.

wokalski

comment created time in 4 days

PR opened graphile/worker

Add support for arbitrary functions in place of the cron pattern

Description

Fixes #218

Please keep in mind that this PR includes some changes. Cron expressions are not any different from any other function you can pass as the "schedule matcher".

Checklist

  • [x] My code matches the project's code style and yarn lint:fix passes.
  • [x] I've added tests for the new feature, and yarn test passes.
  • [ ] I have detailed the new feature in the relevant documentation.
  • [ ] I have added this feature to 'Pending' in the RELEASE_NOTES.md file (if one exists).
  • [ ] If this is a breaking change I've explained why.

I haven't updated the documentation yet because I'm not sure if the implementation is satisfying.

+243 -235

0 comment

4 changed files

pr created time in 6 days

push eventdialohq/worker

Wojtek Czekalski

commit sha 34a08f284915ecc8770b5172724abc95c468aa14

Add support for arbitrary functions in place of the cron pattern Fixes #218

view details

push time in 6 days

create barnchdialohq/worker

branch : cron-pattern-function

created branch time in 6 days

issue commentgraphile/worker

Arbitrary functions for cron patterns

Ok, good idea. And how about the function of which creates the cron pattern matcher specifically? Should I name it cron or cronExpression something else ?

wokalski

comment created time in 7 days

issue commentgraphile/worker

Arbitrary functions for cron patterns

@benjie

  1. What name convention would you prefer for it? I thought about allowing both:
pattern: "* * * * *"

as well as

pattern: cron("* * * * *")

Where cron is a function which parses the cron expression.

Alternatively since the string literal representation is a shorthand for it, I wouldn't have anything against naming this function

cronExpression

If we name it cronExpression I wouldn't have anything against creating a dedicated file for it and moving a bunch of stuff over from crontab.ts.

wokalski

comment created time in 7 days

PullRequestReviewEvent

Pull request review commentgraphile/worker

Add last execution time to _cron payload

 async function scheduleCronJobs(   escapedWorkerSchema: string,   jobsAndIdentifiers: JobAndCronIdentifier[],   ts: string,+  backfilled: boolean, ) {   // Note that `identifier` is guaranteed to be unique for every record   // in `specs`.   await pgPool.query(     `-      with specs as (+      with known_crontabs as (++      ),+      specs as (         select           index,           (json->>'identifier')::text as identifier,           ((json->'job')->>'task')::text as task,-          ((json->'job')->'payload')::json as payload,+          (((json->'job')->'payload')::jsonb || jsonb_build_object(

Ok, please let me know what you think, I'll pause until then.

wokalski

comment created time in 7 days

issue openedgraphile/worker

Arbitrary functions for cron patterns

Feature description

Instead of just passing a cron pattern string we could also enable users to pass a function in the same place (i.e. "* * * * *" would be a shorthand for cron("* * * * *") where the function has to follow the TimeDigest -> boolean interface.

Motivating example

One thing this solves in my specific use case are timezone-dependent patterns. I want to run a job in a given timezone, and make it work consistently even if the time offset changes due to daylight saving.

Supporting development

  • [x] am interested in building this feature myself

created time in 7 days

Pull request review commentgraphile/worker

Add last execution time to _cron payload

 test("backfills if identifier already registered (25h)", () =>       expect(jobs.length).toBeGreaterThanOrEqual(6);       expect(jobs.length).toBeLessThanOrEqual(7);       expect(jobs.every((j) => j.task_identifier === "do_it")).toBe(true);+      expect(+        jobs.every(+          (j) =>+            (j.payload as any)["_cron"]["knownSince"] ==+            knownSince.toISOString(),+        ),+      ).toBe(true);+      expect(+        jobs.every(+          (j) =>+            (j.payload as any)["_cron"]["lastOnTimeExecution"] ==+            lastOnTimeExecution.toISOString(),+        ),+      ).toBe(true);

great feedback.

wokalski

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgraphile/worker

Add last execution time to _cron payload

 function unsafeRoundToMinute(ts: Date, roundUp = false): Date {   return ts; } -function makeJobForItem(-  item: ParsedCronItem,-  ts: string,-  backfilled = false,-): CronJob {+function makeJobForItem(item: ParsedCronItem): CronJob {   return {     task: item.task,-    payload: {-      ...item.payload,-      _cron: {-        ts,-        backfilled,-      },-    },+    payload: item.payload ?? {},

this can be hit - if payload is undefined/null. Then on the DB layer we insert the _cron object inside the payload. If it's null it produces an array of two eleemnts. If it's an object, the _cron value is inserted into it.

wokalski

comment created time in 7 days

Pull request review commentgraphile/worker

Add last execution time to _cron payload

 export interface RunnerOptions extends WorkerPoolOptions { export interface CronJob {   task: string;   payload: {-    _cron: { ts: string; backfilled?: boolean };     [key: string]: unknown;   };   queueName?: string;-  runAt: string;

It's inserted in the DB query. No need to put the same information in multiple places IMO.

wokalski

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgraphile/worker

Add last execution time to _cron payload

 export interface RunnerOptions extends WorkerPoolOptions { export interface CronJob {   task: string;   payload: {-    _cron: { ts: string; backfilled?: boolean };

because the CronJob itself (as returned from the make function) doesn't contain it. In this implementation it's defined on the db layer.

wokalski

comment created time in 7 days

Pull request review commentgraphile/worker

Add last execution time to _cron payload

 async function scheduleCronJobs(   escapedWorkerSchema: string,   jobsAndIdentifiers: JobAndCronIdentifier[],   ts: string,+  backfilled: boolean, ) {   // Note that `identifier` is guaranteed to be unique for every record   // in `specs`.   await pgPool.query(     `-      with specs as (+      with known_crontabs as (++      ),+      specs as (         select           index,           (json->>'identifier')::text as identifier,           ((json->'job')->>'task')::text as task,-          ((json->'job')->'payload')::json as payload,+          (((json->'job')->'payload')::jsonb || jsonb_build_object(+              '_cron', jsonb_build_object(+                'ts', $3::text,+                'backfilled', $4::boolean,+                'lastOnTimeExecution', to_char(+                  known_crontabs.last_execution at time zone 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.MS"Z"'+                ),+                'knownSince', to_char(+                  known_crontabs.known_since at time zone 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS.MS"Z"'+                )+              )+            )+          )::json as payload,           ((json->'job')->>'queueName')::text as queue_name,-          ((json->'job')->>'runAt')::timestamptz as run_at,

It doesn't, it's supplied below.

wokalski

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentgraphile/worker

Add last execution time to _cron payload

 async function scheduleCronJobs(   escapedWorkerSchema: string,   jobsAndIdentifiers: JobAndCronIdentifier[],   ts: string,+  backfilled: boolean, ) {   // Note that `identifier` is guaranteed to be unique for every record   // in `specs`.   await pgPool.query(     `-      with specs as (+      with known_crontabs as (++      ),+      specs as (         select           index,           (json->>'identifier')::text as identifier,           ((json->'job')->>'task')::text as task,-          ((json->'job')->'payload')::json as payload,+          (((json->'job')->'payload')::jsonb || jsonb_build_object(

I think we don't know these. If we did I'd use them for sure.

wokalski

comment created time in 7 days

pull request commentgraphile/worker

Add last execution time to _cron payload

Thanks for the feedback; one quick note: runAt is moved to the database layer fully. The rest of the feedback is valid. I'm pretty sure knownCrontabs are not available at the time when scheduling.

I'm also not sure whether I should put more stuff in JS or in the DB but I believe the less we pass the same arguments to multiple places the better. I.e. the runAt change seems like a nice simplification to me.

wokalski

comment created time in 7 days

PR opened graphile/worker

Add last execution time to _cron payload

Description

Fixes #177 (although doesn't add the info to the event)

This PR adds the last execution and known since dates to the _cron payload. I have cron jobs that run every minute need to run back to back. So even if the server is down for some reason I still want to find the last execution date so that I properly report everything that happened since then.

Not all tests pass because I struggled to make lastOnTimeExecution contain the correct values. I wasn't aware that the CTE is more like a view rather than a table so when backfilling, last_exeuction gets changed as the query executes.

Any help would be appreciated here

Performance impact

unknown

It adds a bit of complexity to the SQL query but also removes a bit of complexity from the JS code.

Security impact

unknown

Checklist

  • [x] My code matches the project's code style and yarn lint:fix passes.
  • [x] I've added tests for the new feature, and yarn test passes.
  • [x] I have detailed the new feature in the relevant documentation.
  • [ ] I have added this feature to 'Pending' in the RELEASE_NOTES.md file (if one exists).
  • [ ] If this is a breaking change I've explained why.
+52 -25

0 comment

4 changed files

pr created time in 8 days

push eventdialohq/worker

Wojtek Czekalski

commit sha a154830e7b03152df4e0a0d668ad0bfc461f028d

Add lastExecution and knownSince to cron payload

view details

push time in 8 days

create barnchdialohq/worker

branch : last-execution-time-cron

created branch time in 8 days

create barnchdialohq/worker

branch : fix-cron-dow-matching

created branch time in 8 days

push eventdialohq/worker

Benjie Gillam

commit sha 1d3178f7bd8cfbe2d0fc54f3e566e4370df71712

docs: add note about distributed crontab (#214)

view details

Benjie Gillam

commit sha 50050b1baa9b0f512ce2d6c53e55f427f0f33af0

feat: expose parseCronItem function (#215) Co-authored-by: Wojtek Czekalski <me@wczekalski.com>

view details

Benjie Gillam

commit sha 2b9e2605f72fd48028fa242dba75b6234a121100

fix(cron): day-of-week match now works correctly (#216) Co-authored-by: Wojtek Czekalski <me@wczekalski.com>

view details

Benjie Gillam

commit sha 52f5471e69ec2810cd068b04abb089c9ea34f4f4

docs: release notes

view details

Benjie Gillam

commit sha 276b385193ba300f26e6390eb1439cbe6a1234d6

0.11.4

view details

push time in 9 days

issue commentgraphile/worker

Cron API stability, 6 months later?

One thing I'd reconsider are the ergonomics of the library mode. The API is rather verbose and exposes the internals in a weird way (i.e. the commandment to always use parseCronItems makes it feel like it shouldn't even be a part of the public API)

andrioid

comment created time in 20 days

pull request commentgraphile/worker

Fix cron not matching time properly for days of week

I find this kind of microoptimisations odd but I understand it.

wokalski

comment created time in 20 days

delete branch dialohq/worker

delete branch : fix-cron-dow-matching

delete time in 20 days