profile
viewpoint
Nelson Liu nelson-liu @stanfordnlp Stanford, California https://nelsonliu.me language x computation. @stanfordnlp CS PhD student. Formerly: @uwnlp @Noahs-ARK / @allenai / @isi-nlp / GSoC @scikit-learn.

allenai/allennlp 9154

An open-source NLP research library, built on PyTorch.

ericmjl/nxviz 283

Visualization Package for NetworkX

nelson-liu/contextual-repr-analysis 201

A toolkit for evaluating the linguistic knowledge and transferability of contextual representations. Code for "Linguistic Knowledge and Transferability of Contextual Representations" (NAACL 2019).

allenai/allennlp-as-a-library-example 108

A simple example for how to build your own model using AllenNLP as a dependency.

codalab/codalab-worksheets 91

A collaborative platform for reproducible research (web interface and CLI).

nelson-liu/cython-crash-course 29

A quick intro to Cython for Python users who don't know C

nelson-liu/ASLSpeak 9

:microphone: DubHacks 2015 project. Decode sign language using the Leap Motion, and speak it!

nelson-liu/BitStation-App 3

:money_with_wings: The MIT Kerberos-integrated social wallet. Winner of BitComp 2014 Improving MIT Award.

allenai/allennlp-feedstock 1

A conda-smithy repository for allennlp.

push eventnelson-liu/codalab-worksheets

Yuqi Jin

commit sha 61c54a8243aa06d8a84955647a0289e2a6a3c58d

Fix 'j k' keyboard shortcut for scroll up and down (#2911) * Check a taget element is visible or not to decide whether to scroll * Address comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 8202f6cd120cc8277a53ef86afcf09b59b91e2b2

Remove extra space between the navbar and the homepage (#2943) * Remove extra space between the navbar and the homepage * Remove 'noWrap' Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Levi Lian

commit sha a682dd5e07d03fcb1c342ee041a9d2d696bf54d1

order search results by user's own worksheets (#2954) * order by user's own worksheets * format * format * format Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 68a066e5ffaa3b3d329f6e00b59996cea1575b31

Provide an adaptive text wrap for the first column in the table (#2923) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Nelson Liu

commit sha fe6bf31f14b527d9b436745dbc826a2efce48c0e

Make AWSBatchWorkerManager boto3 warning stricter and more robust (#2897) * Make AWSBatchWorkerManager boto3 warning stricter and more robust * Blacken * simplify * Update codalab/worker_manager/aws_batch_worker_manager.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 0a80bf11d876bed994e1c87454309066ec6fbf34

Fix flake8 errors in codalab/ (#2844) * test eighth * Undo all resolution for invalid escapes * Undo changing interpret * Undo W605 errors

view details

Yuqi Jin

commit sha c9197db059a7c2146be643ad9fc293bfd4cdb5cd

Better style of the schema blocks (#2941) * Check a taget element is visible or not to decide whether to scroll * Address comments * No spacing between schema blocks Reduce the amount of vertical space taken by each block. * Revert "Address comments" This reverts commit 0a3b16f3d737e6b6843c600fc898c6e499683c8d. * Revert "Check a taget element is visible or not to decide whether to scroll" This reverts commit 200262e939d07ce69ca59b36bc7bdb0d115f6baa. * Update frontend test snapshot * Better style of Schema block button * Left justify the icons * Put the icons in the middle

view details

Varyiabol

commit sha 1cd5fc7121b291b796fafd9fb809f8bde5574655

Resolve following flake8 violations (#2965) * Resolve following flake8 violations * Reorganize ./codalab/rest/interpret.py ignoring F821 * Reword comments

view details

Varyiabol

commit sha 547c89a6068336635e934f4dacaf6ccac7129965

Add flake8 check to pre-commit.sh (#2864)

view details

Levi Lian

commit sha 486474cadd6c4123db5b43148724f755b5751c8f

`uls` search for user stats (#2389) * uls 0.9 * combine methods * refactored get_user * search by time & disk usage * add cli tests * passing the test * fix * non-root users cannot query for restricted fields * code format * fix * root/non-root access documentation * non-root user test cli * format * fixed backend bugs * backend test passed * passed all tests * final * test * final * fix formatting * Update codalab/rest/users.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> * fix * fix * format * format * made it clear that keywords, not schema fields, are accessible * format Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: yipenghe <yipenghe@stanford.edu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 32cd704ef92290fe5cf6af77f86de144c3f08c77

Fix More Flake8 Violations (#2975)

view details

Nelson Liu

commit sha 3dadfb6ca5a1c4eb1488f8f2f0705a0ac45f244c

Better handle EC2 spot interruptions in AWS Batch WorkerManager (#2977) * Better handle EC2 spot interruptions in AWS Batch WorkerManager * Add 5 second sleep so we don't spam pkill * Use instance-action endpoint * Don't send SIGTERM in loop * Typo fix

view details

Nelson Liu

commit sha 060cccb7ddb437c0ff357530daedaace58e7a1a9

Add boto3 to requirements.txt

view details

Nelson Liu

commit sha 054d9d6d35e9cc52dfcb44a5d7a25c62979942c5

Upload to s3 instead of bundle service

view details

Nelson Liu

commit sha 85e8de3c0618caf997998406fdb48da5db11f23d

Modify workermanager image

view details

Nelson Liu

commit sha 28011e7e7dfd8bec6d13e3a3c5844bd449ce15a4

Fix dockerhub username

view details

Nelson Liu

commit sha 915c3b6fcf7ad49854ed0dfcb47ff305a5866108

Lint

view details

Nelson Liu

commit sha 9739e05fe51dd5d0e703da664efc018e0e145619

Add missing boto3 import

view details

Nelson Liu

commit sha 836e0c42915de01dcf77cd7f33f17e03f9fb1dee

Add missing imoprt

view details

Nelson Liu

commit sha 61bd81a74c36ae2efc6d5e5dd286f25d13f839b0

Try uploading stderr and stdout only

view details

push time in a day

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 3986d1bd1782b2ed12a2221ced801461ad1b9296

Add CodalabManagerState classes, with support for json and sqlite3

view details

push time in 2 days

issue commentkidd/org-gcal.el

no errors but calendar buffers are still empty after fetch/sync

@swapnilsm 's fix worked for me as well---thanks for sharing!

It might be applicable to others like me who are running doom emacs. In my case, doom had set org-id-locations-file, but it / its folder was nonexistent.

wminshew

comment created time in 2 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 3dadfb6ca5a1c4eb1488f8f2f0705a0ac45f244c

Better handle EC2 spot interruptions in AWS Batch WorkerManager (#2977) * Better handle EC2 spot interruptions in AWS Batch WorkerManager * Add 5 second sleep so we don't spam pkill * Use instance-action endpoint * Don't send SIGTERM in loop * Typo fix

view details

Nelson Liu

commit sha 9dbe08652d73d65866c5aa30991ec44551eaf033

Add CodalabManagerState classes, with support for json and sqlite3

view details

push time in 2 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 3dadfb6ca5a1c4eb1488f8f2f0705a0ac45f244c

Better handle EC2 spot interruptions in AWS Batch WorkerManager (#2977) * Better handle EC2 spot interruptions in AWS Batch WorkerManager * Add 5 second sleep so we don't spam pkill * Use instance-action endpoint * Don't send SIGTERM in loop * Typo fix

view details

push time in 2 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha 3dadfb6ca5a1c4eb1488f8f2f0705a0ac45f244c

Better handle EC2 spot interruptions in AWS Batch WorkerManager (#2977) * Better handle EC2 spot interruptions in AWS Batch WorkerManager * Add 5 second sleep so we don't spam pkill * Use instance-action endpoint * Don't send SIGTERM in loop * Typo fix

view details

push time in 2 days

delete branch codalab/codalab-worksheets

delete branch : ec2_interruption

delete time in 2 days

PR merged codalab/codalab-worksheets

Reviewers
Better handle EC2 spot interruptions in AWS Batch WorkerManager

(Continuation of #2976 , opening from a branch on codalab/codalab-worksheets so the tag is pushed to docker hub)

Reasons for making this change

When EC2 spot instances are preempted, they're given 2 minutes to clean up. These cleanup signals are sent via an AWS web endpoint (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#spot-instance-termination-notices). Then, the container terminates.

Right now, we don't catch these interruption notices, so jobs running on AWS workers will go into worker_offline state instead of being restaged. This PR adds a script to (1) catch the EC2 preemption notices and (2) send a SIGTERM to all CodaLab workers when it sees one. When the CodaLab worker gets a SIGTERM, it should restage its jobs (if applicable) / clean up and prepare for a graceful exit.

+31 -2

1 comment

3 changed files

nelson-liu

pr closed time in 2 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha e368e504d9637ff93a1ad3a430825aa38a2181b1

Typo fix

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha 057ed27f244ed1454d61feef6ecc992fcb99f1a7

Don't send SIGTERM in loop

view details

push time in 3 days

pull request commentcodalab/codalab-worksheets

Better handle EC2 spot interruptions in AWS Batch WorkerManager

I tested this branch as-is before ashwin's comment, and it seemed to work (according to the Batch logs, that is)

nelson-liu

comment created time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha 7fc243f8b7283117e3c90320fb425fabf0052af7

Use instance-action endpoint

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha 02571c868a482d97d35b1342ad883af012a871f0

Add 5 second sleep so we don't spam pkill

view details

push time in 3 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 4130eeeb20afd0e97ca197f533aae4e4f2ff98da

Add CodalabManagerState classes, with support for json and sqlite3

view details

Nelson Liu

commit sha 3860c6b926289c380ebdff3e597453f6aaa2e64f

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha d16682ba110d3f5e958d2b1e0c5636c5c311cc6c

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha bcc9404109b382e8730040f9c773830cab078687

Add CodalabManagerState classes, with support for json and sqlite3

view details

Nelson Liu

commit sha d9aba6838088183a0ac7564df47f7f1ad4600767

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha ecd6fe0844985ecbfbf87358d3a843ff0f793ec2

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 4ee5bad9fc69d04c27e82a31e9a5d7bd7efe1e8b

Add CodalabManagerState classes, with support for json and sqlite3

view details

Nelson Liu

commit sha 130bd279460048aae3329717fc31643e9841e670

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha 16e0c49cbb95d4807190f3cbe2accf958226320b

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha 63c500528258422d8926dcf08b0a76a4d51c5f69

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha eeb42b712eec4e51d8407c305f959d1cccc943bd

AWS Batch doesn't interpret shell commands

view details

push time in 3 days

create barnchnelson-liu/codalab-worksheets

branch : ec2_interruption

created branch time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha 3258531107b6c3d3f8e34af723eb1d24e329d81c

Add curl to worker Dockerfile

view details

push time in 3 days

push eventnelson-liu/codalab-worksheets

push time in 3 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha c244686ad1f56651825af2eaf03193d8b6cf1a42

Add cURL to worker Dockerfile

view details

push time in 3 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 30c5d6131800800a9ec7762481493ab43fdebda4

Add CodalabManagerState classes, with support for json and sqlite3

view details

Nelson Liu

commit sha 8189447ccd3efe50014dc1b28253d3e0751f4c27

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventcodalab/codalab-worksheets

Nelson Liu

commit sha fb06d4dec8fbf5cdead89923892c0be0cd24fcd8

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 3 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha bb0cb50bca98a26e2b2d6e7676a17a20d63e2364

Add CodalabManagerState classes, with support for json and sqlite3

view details

Nelson Liu

commit sha 79b43756d9ba47a6afb79287a9f79b390757d2cc

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 4 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha d67bdf90cbbadb6624c67c5c31360ea6ec42e1b6

Add CodalabManagerState classes, with support for json and sqlite3

view details

push time in 4 days

push eventnelson-liu/codalab-worksheets

Yuqi Jin

commit sha 61c54a8243aa06d8a84955647a0289e2a6a3c58d

Fix 'j k' keyboard shortcut for scroll up and down (#2911) * Check a taget element is visible or not to decide whether to scroll * Address comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 8202f6cd120cc8277a53ef86afcf09b59b91e2b2

Remove extra space between the navbar and the homepage (#2943) * Remove extra space between the navbar and the homepage * Remove 'noWrap' Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Levi Lian

commit sha a682dd5e07d03fcb1c342ee041a9d2d696bf54d1

order search results by user's own worksheets (#2954) * order by user's own worksheets * format * format * format Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 68a066e5ffaa3b3d329f6e00b59996cea1575b31

Provide an adaptive text wrap for the first column in the table (#2923) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Nelson Liu

commit sha fe6bf31f14b527d9b436745dbc826a2efce48c0e

Make AWSBatchWorkerManager boto3 warning stricter and more robust (#2897) * Make AWSBatchWorkerManager boto3 warning stricter and more robust * Blacken * simplify * Update codalab/worker_manager/aws_batch_worker_manager.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 0a80bf11d876bed994e1c87454309066ec6fbf34

Fix flake8 errors in codalab/ (#2844) * test eighth * Undo all resolution for invalid escapes * Undo changing interpret * Undo W605 errors

view details

Yuqi Jin

commit sha c9197db059a7c2146be643ad9fc293bfd4cdb5cd

Better style of the schema blocks (#2941) * Check a taget element is visible or not to decide whether to scroll * Address comments * No spacing between schema blocks Reduce the amount of vertical space taken by each block. * Revert "Address comments" This reverts commit 0a3b16f3d737e6b6843c600fc898c6e499683c8d. * Revert "Check a taget element is visible or not to decide whether to scroll" This reverts commit 200262e939d07ce69ca59b36bc7bdb0d115f6baa. * Update frontend test snapshot * Better style of Schema block button * Left justify the icons * Put the icons in the middle

view details

Varyiabol

commit sha 1cd5fc7121b291b796fafd9fb809f8bde5574655

Resolve following flake8 violations (#2965) * Resolve following flake8 violations * Reorganize ./codalab/rest/interpret.py ignoring F821 * Reword comments

view details

Varyiabol

commit sha 547c89a6068336635e934f4dacaf6ccac7129965

Add flake8 check to pre-commit.sh (#2864)

view details

Levi Lian

commit sha 486474cadd6c4123db5b43148724f755b5751c8f

`uls` search for user stats (#2389) * uls 0.9 * combine methods * refactored get_user * search by time & disk usage * add cli tests * passing the test * fix * non-root users cannot query for restricted fields * code format * fix * root/non-root access documentation * non-root user test cli * format * fixed backend bugs * backend test passed * passed all tests * final * test * final * fix formatting * Update codalab/rest/users.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> * fix * fix * format * format * made it clear that keywords, not schema fields, are accessible * format Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: yipenghe <yipenghe@stanford.edu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 32cd704ef92290fe5cf6af77f86de144c3f08c77

Fix More Flake8 Violations (#2975)

view details

Nelson Liu

commit sha 02c6d873dc17ead00076f4f827212e2d1dfc1a1c

Add CodalabManagerState classes, with support for json and sqlite3

view details

Nelson Liu

commit sha 2a4e73233ba8b736a2bc83e73e94af7e7dafffbb

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 4 days

push eventnelson-liu/codalab-worksheets

yuqijin

commit sha 04fff6e34c00c2de79bf66b97cc450d617eb9741

Bump to version 0.5.23

view details

Tony Lee

commit sha 830596f16522ef84dec9a9a1539a06d5ba218832

fix --job-filter (#2945)

view details

Ashwin Ramaswami

commit sha 9d286ec7f81c04e0a127884c5c0d023093c9f364

BundleManager tests (#2795) * reorganize tests, add webtest for rest api tests * Add frontend tests * Update some documentation * fix unit test run command * add rest api tests * update test * fix test files location * black format * fix tests * black format * refactor, fix rest api tests * add testing philosophy * Update snapshots * Add code coverage checks * update nomenclature * fix unittest * cleanup * fix path * fix test path * fix test * Fix tests, make running unit tests easier * try test fixes * add chown command * fix coverage * fix test * fix test again * fix test * fix test * fix * add sqlite model * Add bundle manager tests * fix sqlite db to work across threads * more tests, fix issue where cpus defaults to 1 * Add BundleManagerFailUnresponsiveBundlesTest * more tests * more tests * split test into multiple files * Update codalab/server/bundle_manager.py * fix tests * undo * more abstractions * refactor * freezegun * refactor * add comments * update names * format * typing * Fix typing * refactor * more refactoring * Add comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Ashwin Ramaswami

commit sha 5c4b76bc594c0f26176fa82b3ea27cf2a845d79d

Run github actions on all PRs, even those that are not to master (#2956)

view details

Ashwin Ramaswami

commit sha 414c2c3d186baf3f43ff87f166e86c59b8f222d3

Merge pull request #2932 from codalab/rc0.5.23 Bump to version 0.5.23

view details

Yuqi Jin

commit sha 61c54a8243aa06d8a84955647a0289e2a6a3c58d

Fix 'j k' keyboard shortcut for scroll up and down (#2911) * Check a taget element is visible or not to decide whether to scroll * Address comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 8202f6cd120cc8277a53ef86afcf09b59b91e2b2

Remove extra space between the navbar and the homepage (#2943) * Remove extra space between the navbar and the homepage * Remove 'noWrap' Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Levi Lian

commit sha a682dd5e07d03fcb1c342ee041a9d2d696bf54d1

order search results by user's own worksheets (#2954) * order by user's own worksheets * format * format * format Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 68a066e5ffaa3b3d329f6e00b59996cea1575b31

Provide an adaptive text wrap for the first column in the table (#2923) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Nelson Liu

commit sha fe6bf31f14b527d9b436745dbc826a2efce48c0e

Make AWSBatchWorkerManager boto3 warning stricter and more robust (#2897) * Make AWSBatchWorkerManager boto3 warning stricter and more robust * Blacken * simplify * Update codalab/worker_manager/aws_batch_worker_manager.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 0a80bf11d876bed994e1c87454309066ec6fbf34

Fix flake8 errors in codalab/ (#2844) * test eighth * Undo all resolution for invalid escapes * Undo changing interpret * Undo W605 errors

view details

Yuqi Jin

commit sha c9197db059a7c2146be643ad9fc293bfd4cdb5cd

Better style of the schema blocks (#2941) * Check a taget element is visible or not to decide whether to scroll * Address comments * No spacing between schema blocks Reduce the amount of vertical space taken by each block. * Revert "Address comments" This reverts commit 0a3b16f3d737e6b6843c600fc898c6e499683c8d. * Revert "Check a taget element is visible or not to decide whether to scroll" This reverts commit 200262e939d07ce69ca59b36bc7bdb0d115f6baa. * Update frontend test snapshot * Better style of Schema block button * Left justify the icons * Put the icons in the middle

view details

Varyiabol

commit sha 1cd5fc7121b291b796fafd9fb809f8bde5574655

Resolve following flake8 violations (#2965) * Resolve following flake8 violations * Reorganize ./codalab/rest/interpret.py ignoring F821 * Reword comments

view details

Varyiabol

commit sha 547c89a6068336635e934f4dacaf6ccac7129965

Add flake8 check to pre-commit.sh (#2864)

view details

Levi Lian

commit sha 486474cadd6c4123db5b43148724f755b5751c8f

`uls` search for user stats (#2389) * uls 0.9 * combine methods * refactored get_user * search by time & disk usage * add cli tests * passing the test * fix * non-root users cannot query for restricted fields * code format * fix * root/non-root access documentation * non-root user test cli * format * fixed backend bugs * backend test passed * passed all tests * final * test * final * fix formatting * Update codalab/rest/users.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> * fix * fix * format * format * made it clear that keywords, not schema fields, are accessible * format Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: yipenghe <yipenghe@stanford.edu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 32cd704ef92290fe5cf6af77f86de144c3f08c77

Fix More Flake8 Violations (#2975)

view details

Nelson Liu

commit sha 97ffd0ae1bde3d75d1a8d9bf79f3b076f2cb2cdc

Add CodalabManagerState classes, with support for json and sqlite3

view details

push time in 4 days

delete branch nelson-liu/codalab-worksheets

delete branch : ec2_interruption

delete time in 4 days

PR closed codalab/codalab-worksheets

Reviewers
Better handle EC2 spot interruptions in AWS Batch WorkerManager

Reasons for making this change

When EC2 spot instances are preempted, they're given 2 minutes to clean up. These cleanup signals are sent via an AWS web endpoint (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#spot-instance-termination-notices). Then, the container terminates.

Right now, we don't catch these interruption notices, so jobs running on AWS workers will go into worker_offline state instead of being restaged. This PR adds a script to (1) catch the EC2 preemption notices and (2) send a SIGTERM to all CodaLab workers when it sees one. When the CodaLab worker gets a SIGTERM, it should restage its jobs (if applicable) / clean up and prepare for a graceful exit.

+26 -1

1 comment

3 changed files

nelson-liu

pr closed time in 4 days

pull request commentcodalab/codalab-worksheets

Better handle EC2 spot interruptions in AWS Batch WorkerManager

superseded by #2977

nelson-liu

comment created time in 4 days

PR opened codalab/codalab-worksheets

Better handle EC2 spot interruptions in AWS Batch WorkerManager

(Continuation of #2976 , opening from a branch on codalab/codalab-worksheets so the tag is pushed to docker hub)

Reasons for making this change

When EC2 spot instances are preempted, they're given 2 minutes to clean up. These cleanup signals are sent via an AWS web endpoint (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#spot-instance-termination-notices). Then, the container terminates.

Right now, we don't catch these interruption notices, so jobs running on AWS workers will go into worker_offline state instead of being restaged. This PR adds a script to (1) catch the EC2 preemption notices and (2) send a SIGTERM to all CodaLab workers when it sees one. When the CodaLab worker gets a SIGTERM, it should restage its jobs (if applicable) / clean up and prepare for a graceful exit.

+26 -1

0 comment

3 changed files

pr created time in 4 days

create barnchcodalab/codalab-worksheets

branch : ec2_interruption

created branch time in 4 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 265f875ab9baa53e73fa6e719ca6cfd6c90f59a7

Better handle EC2 spot interruptions in AWS Batch WorkerManager

view details

push time in 4 days

push eventnelson-liu/codalab-worksheets

Yuqi Jin

commit sha 61c54a8243aa06d8a84955647a0289e2a6a3c58d

Fix 'j k' keyboard shortcut for scroll up and down (#2911) * Check a taget element is visible or not to decide whether to scroll * Address comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 8202f6cd120cc8277a53ef86afcf09b59b91e2b2

Remove extra space between the navbar and the homepage (#2943) * Remove extra space between the navbar and the homepage * Remove 'noWrap' Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Levi Lian

commit sha a682dd5e07d03fcb1c342ee041a9d2d696bf54d1

order search results by user's own worksheets (#2954) * order by user's own worksheets * format * format * format Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 68a066e5ffaa3b3d329f6e00b59996cea1575b31

Provide an adaptive text wrap for the first column in the table (#2923) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Nelson Liu

commit sha fe6bf31f14b527d9b436745dbc826a2efce48c0e

Make AWSBatchWorkerManager boto3 warning stricter and more robust (#2897) * Make AWSBatchWorkerManager boto3 warning stricter and more robust * Blacken * simplify * Update codalab/worker_manager/aws_batch_worker_manager.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 0a80bf11d876bed994e1c87454309066ec6fbf34

Fix flake8 errors in codalab/ (#2844) * test eighth * Undo all resolution for invalid escapes * Undo changing interpret * Undo W605 errors

view details

Yuqi Jin

commit sha c9197db059a7c2146be643ad9fc293bfd4cdb5cd

Better style of the schema blocks (#2941) * Check a taget element is visible or not to decide whether to scroll * Address comments * No spacing between schema blocks Reduce the amount of vertical space taken by each block. * Revert "Address comments" This reverts commit 0a3b16f3d737e6b6843c600fc898c6e499683c8d. * Revert "Check a taget element is visible or not to decide whether to scroll" This reverts commit 200262e939d07ce69ca59b36bc7bdb0d115f6baa. * Update frontend test snapshot * Better style of Schema block button * Left justify the icons * Put the icons in the middle

view details

Varyiabol

commit sha 1cd5fc7121b291b796fafd9fb809f8bde5574655

Resolve following flake8 violations (#2965) * Resolve following flake8 violations * Reorganize ./codalab/rest/interpret.py ignoring F821 * Reword comments

view details

Varyiabol

commit sha 547c89a6068336635e934f4dacaf6ccac7129965

Add flake8 check to pre-commit.sh (#2864)

view details

Levi Lian

commit sha 486474cadd6c4123db5b43148724f755b5751c8f

`uls` search for user stats (#2389) * uls 0.9 * combine methods * refactored get_user * search by time & disk usage * add cli tests * passing the test * fix * non-root users cannot query for restricted fields * code format * fix * root/non-root access documentation * non-root user test cli * format * fixed backend bugs * backend test passed * passed all tests * final * test * final * fix formatting * Update codalab/rest/users.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> * fix * fix * format * format * made it clear that keywords, not schema fields, are accessible * format Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: yipenghe <yipenghe@stanford.edu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 32cd704ef92290fe5cf6af77f86de144c3f08c77

Fix More Flake8 Violations (#2975)

view details

push time in 4 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 6d3a15e821165b098c124cd96200ae80c3dbf11b

Add some more documentation to ec2 spot pre-emption script

view details

push time in 4 days

push eventnelson-liu/codalab-worksheets

Yuqi Jin

commit sha 61c54a8243aa06d8a84955647a0289e2a6a3c58d

Fix 'j k' keyboard shortcut for scroll up and down (#2911) * Check a taget element is visible or not to decide whether to scroll * Address comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 8202f6cd120cc8277a53ef86afcf09b59b91e2b2

Remove extra space between the navbar and the homepage (#2943) * Remove extra space between the navbar and the homepage * Remove 'noWrap' Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Levi Lian

commit sha a682dd5e07d03fcb1c342ee041a9d2d696bf54d1

order search results by user's own worksheets (#2954) * order by user's own worksheets * format * format * format Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Yuqi Jin

commit sha 68a066e5ffaa3b3d329f6e00b59996cea1575b31

Provide an adaptive text wrap for the first column in the table (#2923) Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Nelson Liu

commit sha fe6bf31f14b527d9b436745dbc826a2efce48c0e

Make AWSBatchWorkerManager boto3 warning stricter and more robust (#2897) * Make AWSBatchWorkerManager boto3 warning stricter and more robust * Blacken * simplify * Update codalab/worker_manager/aws_batch_worker_manager.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 0a80bf11d876bed994e1c87454309066ec6fbf34

Fix flake8 errors in codalab/ (#2844) * test eighth * Undo all resolution for invalid escapes * Undo changing interpret * Undo W605 errors

view details

Yuqi Jin

commit sha c9197db059a7c2146be643ad9fc293bfd4cdb5cd

Better style of the schema blocks (#2941) * Check a taget element is visible or not to decide whether to scroll * Address comments * No spacing between schema blocks Reduce the amount of vertical space taken by each block. * Revert "Address comments" This reverts commit 0a3b16f3d737e6b6843c600fc898c6e499683c8d. * Revert "Check a taget element is visible or not to decide whether to scroll" This reverts commit 200262e939d07ce69ca59b36bc7bdb0d115f6baa. * Update frontend test snapshot * Better style of Schema block button * Left justify the icons * Put the icons in the middle

view details

Varyiabol

commit sha 1cd5fc7121b291b796fafd9fb809f8bde5574655

Resolve following flake8 violations (#2965) * Resolve following flake8 violations * Reorganize ./codalab/rest/interpret.py ignoring F821 * Reword comments

view details

Varyiabol

commit sha 547c89a6068336635e934f4dacaf6ccac7129965

Add flake8 check to pre-commit.sh (#2864)

view details

Levi Lian

commit sha 486474cadd6c4123db5b43148724f755b5751c8f

`uls` search for user stats (#2389) * uls 0.9 * combine methods * refactored get_user * search by time & disk usage * add cli tests * passing the test * fix * non-root users cannot query for restricted fields * code format * fix * root/non-root access documentation * non-root user test cli * format * fixed backend bugs * backend test passed * passed all tests * final * test * final * fix formatting * Update codalab/rest/users.py Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> * fix * fix * format * format * made it clear that keywords, not schema fields, are accessible * format Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com> Co-authored-by: yipenghe <yipenghe@stanford.edu> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Varyiabol

commit sha 32cd704ef92290fe5cf6af77f86de144c3f08c77

Fix More Flake8 Violations (#2975)

view details

Nelson Liu

commit sha fc1811ac74ee78d628af15cb8494a299ddd2ff4e

Merge branch 'master' into ec2_interruption

view details

push time in 4 days

push eventcodalab/codalab-worksheets

Varyiabol

commit sha 32cd704ef92290fe5cf6af77f86de144c3f08c77

Fix More Flake8 Violations (#2975)

view details

push time in 4 days

delete branch codalab/codalab-worksheets

delete branch : fixMoreFlake8Violations

delete time in 4 days

PR merged codalab/codalab-worksheets

Reviewers
Fix More Flake8 Violations

Reasons for making this change

Some flake8 violations are still there in the master <!-- Add a reason for making this change here. -->

Related issues

#2660 <!-- Add a reference to issues resolved, if applicable (for example, "fixes #1"). -->

Screenshots

<!-- Add screenshots, if necessary -->

Checklist

  • [ ] I've added a screenshot of the changes, if this is a frontend change
  • [ ] I've added and/or updated tests, if this is a backend change
  • [ ] I've run the pre-commit.sh script
  • [ ] I've updated docs, if needed
+2 -4

0 comment

2 changed files

yiboliu

pr closed time in 4 days

PullRequestReviewEvent

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 7a0c7cfd65522a2f13ad40159830b8d6d0c987b1

Lint

view details

push time in 4 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha f4163d8dd6337f2626749f4b2b2f9dd3c1fe4a83

pgrep | xargs kill is more portable than killall -f

view details

push time in 4 days

PR opened codalab/codalab-worksheets

Better handle EC2 spot interruptions in AWS Batch WorkerManager

Reasons for making this change

When EC2 spot instances are preempted, they're given 2 minutes to clean up. These cleanup signals are sent via an AWS web endpoint (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html#spot-instance-termination-notices). Then, the container terminates.

Right now, we don't catch these interruption notices, so jobs running on AWS workers will go into worker_offline state instead of being restaged. This PR adds a script to (1) catch the EC2 preemption notices and (2) send a SIGTERM to all CodaLab workers when it sees one. When the CodaLab worker gets a SIGTERM, it should restage its jobs (if applicable) / clean up and prepare for a graceful exit.

+18 -1

0 comment

3 changed files

pr created time in 4 days

create barnchnelson-liu/codalab-worksheets

branch : ec2_interruption

created branch time in 4 days

PR opened StanfordVL/sail-blog

Fix typo in README
+1 -1

0 comment

1 changed file

pr created time in 6 days

push eventnelson-liu/sail-blog

Nelson Liu

commit sha 64344383b0664b85de9f0e2ac82cee98d9f62187

Fix typo in README

view details

push time in 6 days

push eventconda-forge/pyhocon-feedstock

conda-forge-admin

commit sha b9d389012f3b18c71d928407286a0af2270fe740

[ci skip] ***NO_CI*** added bot automerge

view details

Nelson Liu

commit sha 004ec71609121a1003c62b8c461eaecee11a2661

Merge pull request #23 from conda-forge-linter/conda_forge_admin_22 [ci skip] [cf admin skip] ***NO_CI*** adding bot automerge

view details

push time in 7 days

PR merged conda-forge/pyhocon-feedstock

Reviewers
[ci skip] [cf admin skip] ***NO_CI*** adding bot automerge

Hi! This is the friendly automated conda-forge-webservice.

I've added bot automerge as instructed in #22.

Merge this PR to enable bot automerging.

Fixes #22

+2 -0

0 comment

1 changed file

conda-forge-linter

pr closed time in 7 days

issue closedconda-forge/pyhocon-feedstock

@conda-forge-admin,please add bot automerge

<!-- Thanks for reporting your issue. Please fill out the sections below. --> Issue:

<br/> Environment (<code>conda list</code>): <details>

$ conda list

</details>

<br/> Details about <code>conda</code> and system ( <code>conda info</code> ): <details>

$ conda info

</details>

closed time in 7 days

nelson-liu

issue openedconda-forge/pyhocon-feedstock

@conda-forge-admin,please add bot automerge

<!-- Thanks for reporting your issue. Please fill out the sections below. --> Issue:

<br/> Environment (<code>conda list</code>): <details>

$ conda list

</details>

<br/> Details about <code>conda</code> and system ( <code>conda info</code> ): <details>

$ conda info

</details>

created time in 7 days

push eventconda-forge/pyhocon-feedstock

regro-cf-autotick-bot

commit sha 895f45cecaa876225bc68332005d36cae157ece8

updated v0.3.56

view details

regro-cf-autotick-bot

commit sha 50b565c9bb332b8a3a438f3d937aa174601fb5bc

MNT: Re-rendered with conda-build 3.20.4, conda-smithy 3.8.2, and conda-forge-pinning 2020.10.14.13.15.42

view details

Nelson Liu

commit sha 5ee5d14fb5fb3fd7f2b075c85a12b6ac9b5d7935

Merge pull request #21 from regro-cf-autotick-bot/0.3.56_h265f05 pyhocon v0.3.56

view details

push time in 7 days

PR merged conda-forge/pyhocon-feedstock

Reviewers
pyhocon v0.3.56

It is very likely that the current package version for this feedstock is out of date. Notes for merging this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version. Checklist before merging this PR:
  • [ ] Dependencies have been updated if changed: see upstream
  • [ ] Tests have passed
  • [ ] Updated license if changed and license_file is packaged

Note that the bot will stop issuing PRs if more than 3 Version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.

NEW: If you want these PRs to be merged automatically, make an issue with <code>@conda-forge-admin,</code>please add bot automerge in the title and merge the resulting PR. This command will add our new bot automerge feature to your feedstock!

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase <code>@<space/>conda-forge-admin, please rerun bot</code> in a PR comment to have the conda-forge-admin add it for you.

<sub>This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. If you would like a local version of this bot, you might consider using rever. Rever is a tool for automating software releases and forms the backbone of the bot's conda-forge PRing capability. Rever is both conda (conda install -c conda-forge rever) and pip (pip install re-ver) installable. Finally, feel free to drop us a line if there are any issues! This PR was generated by , please use this URL for debugging</sub>

Dependency Analysis

Please note that this analysis is highly experimental. The aim here is to make maintenance easier by inspecting the package's dependencies. Importantly this analysis does not support optional dependencies, please double check those before making changes. If you do not want hinting of this kind ever please add bot: inspection: false to your conda-forge.yml. If you encounter issues with this feature please ping the bot team conda-forge/bot.

Analysis of the source code shows no discrepancy between the library's imports and the package's stated requirements in the meta.yaml.

+56 -14

1 comment

8 changed files

regro-cf-autotick-bot

pr closed time in 7 days

pull request commentconda-forge/pyhocon-feedstock

pyhocon v0.3.56

@conda-forge-admin, please rerun bot

regro-cf-autotick-bot

comment created time in 7 days

push eventregro-cf-autotick-bot/pyhocon-feedstock

regro-cf-autotick-bot

commit sha 0e38b3d26ccee288d1dbb2f4b49241c3c834a463

updated v0.3.55

view details

regro-cf-autotick-bot

commit sha 44e6aed3791c59e012677fde7ca1da9c7b1e3ff4

MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.3, and conda-forge-pinning 2020.05.25.23.01.15

view details

Nelson Liu

commit sha da9f546e9b69f11e6a4b6075e38ae300ea1c5f83

Add asset to meta.yaml

view details

Nelson Liu

commit sha e2805d8a11b654ac40caadb54e5e42f93aa14a44

Add setuptools

view details

Nelson Liu

commit sha 69a4cf61cdc23c9177e638cf89b642f142fd196f

Merge pull request #19 from regro-cf-autotick-bot/0.3.55_ha1b45f pyhocon v0.3.55

view details

Nelson Liu

commit sha 37f0dcf2ed23561e1095ee8f615170d60bed5064

Merge branch 'master' into 0.3.56_h274112

view details

push time in 7 days

push eventconda-forge/pyhocon-feedstock

regro-cf-autotick-bot

commit sha 0e38b3d26ccee288d1dbb2f4b49241c3c834a463

updated v0.3.55

view details

regro-cf-autotick-bot

commit sha 44e6aed3791c59e012677fde7ca1da9c7b1e3ff4

MNT: Re-rendered with conda-build 3.19.2, conda-smithy 3.7.3, and conda-forge-pinning 2020.05.25.23.01.15

view details

Nelson Liu

commit sha da9f546e9b69f11e6a4b6075e38ae300ea1c5f83

Add asset to meta.yaml

view details

Nelson Liu

commit sha e2805d8a11b654ac40caadb54e5e42f93aa14a44

Add setuptools

view details

Nelson Liu

commit sha 69a4cf61cdc23c9177e638cf89b642f142fd196f

Merge pull request #19 from regro-cf-autotick-bot/0.3.55_ha1b45f pyhocon v0.3.55

view details

push time in 7 days

PR merged conda-forge/pyhocon-feedstock

Reviewers
pyhocon v0.3.55

It is very likely that the current package version for this feedstock is out of date. Notes for merging this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version. Checklist before merging this PR:
  • [x] Dependencies have been updated if changed: see upstream
  • [x] Tests have passed
  • [x] Updated license if changed and license_file is packaged

Note that the bot will stop issuing PRs if more than 3 Version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.

NEW: If you want these PRs to be merged automatically, make an issue with <code>@conda-forge-admin,</code>please add bot automerge in the title and merge the resulting PR. This command will add our new bot automerge feature to your feedstock!

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase <code>@<space/>conda-forge-admin, please rerun bot</code> in a PR comment to have the conda-forge-admin add it for you.

<sub>This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. If you would like a local version of this bot, you might consider using rever. Rever is a tool for automating software releases and forms the backbone of the bot's conda-forge PRing capability. Rever is both conda (conda install -c conda-forge rever) and pip (pip install re-ver) installable. Finally, feel free to drop us a line if there are any issues! This PR was generated by https://circleci.com/gh/regro/circle_worker/19559, please use this URL for debugging</sub>

+28 -19

1 comment

5 changed files

regro-cf-autotick-bot

pr closed time in 7 days

push eventregro-cf-autotick-bot/pyhocon-feedstock

Nelson Liu

commit sha e2805d8a11b654ac40caadb54e5e42f93aa14a44

Add setuptools

view details

push time in 7 days

push eventregro-cf-autotick-bot/pyhocon-feedstock

Nelson Liu

commit sha da9f546e9b69f11e6a4b6075e38ae300ea1c5f83

Add asset to meta.yaml

view details

push time in 7 days

issue commentpytorch/pytorch

Version 1.3 no longer supporting Tesla K40m?

If it helps, I put up some K40-compatible pip binaries at https://nelsonliu.me/files/pytorch/whl/torch_stable.html . Versions 1.3.1 to 1.6.0, hoping to keep them updated for new releases.

You can pip-install them with (change desired versions as necessary):

pip install torch==1.3.1+cu92 -f https://nelsonliu.me/files/pytorch/whl/torch_stable.html

I tested all of these binaries (except the CUDA 10.2 ones, hoping to get to those soon) by running the word-level language modeling example on a K40 and manually verifying that the perplexity was the same within versions and generally reasonable.

JamesOwers

comment created time in 8 days

pull request commentconda-forge/staged-recipes

Add recipe for asset

Looks like this is ready to go, CI-wise

nelson-liu

comment created time in 8 days

push eventnelson-liu/staged-recipes

Nelson Liu

commit sha ca3370eb275f6c43a88708a35fe9a861de17dfd4

Add recipe for asset

view details

push time in 8 days

push eventnelson-liu/staged-recipes

Nelson Liu

commit sha 36615f3217241a825154e3c0d92bf533b61ef426

Trigger CI

view details

push time in 8 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha cdb96d86b83c7132049a82a1d7587797283c0314

Move import

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha d88485ef31a54b101a0daefdeda2986e101557a8

Fix uploading stderr/stdout to use BundleServiceClient

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 0d9333e945693659bc6c4fbcf69e973a2e9743e2

Hopefully one last final import

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha b291cd7895c251993689e93d7ef0d3392bc3b529

Capitalize arg name

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha fe0b6614ba1cd6add13fb2277c0824c8f42f9f82

Import closing

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 787daa3348816b9fb6dbbdc7994d2a6755372985

Try uploading stderr and stdout only

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha f9aedecd63fb23830ef105beaac01943f481bede

Add missing imoprt

view details

push time in 9 days

pull request commentconda-forge/staged-recipes

Add recipe for asset

sounds good, thanks @synapticarbors !

nelson-liu

comment created time in 9 days

pull request commentnelson-liu/codalab-worksheets

Upload to aws S3 instead of bundleservice

(it's expected that this PR fails tests, since the ability to upload to the bundle service is removed)

nelson-liu

comment created time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 70d451e66d94868fbec9c0d5eee08c08c486c54a

Add missing boto3 import

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 96b27dc76049ab9ebe73509c290954f61edb42db

Lint

view details

push time in 9 days

push eventnelson-liu/codalab-worksheets

Nelson Liu

commit sha 1b109fbbd1f1e862982a7f89cc1e6372d474e80d

Fix dockerhub username

view details

push time in 9 days

pull request commentnelson-liu/codalab-worksheets

Upload to aws S3 instead of bundleservice

building the docker image:

docker build . -t nfliu/s3-cl-worker -f docker/dockerfiles/Dockerfile.worker
docker push -f nfliu/s3-cl-worker
nelson-liu

comment created time in 9 days

PR opened nelson-liu/codalab-worksheets

Upload to aws S3 instead of bundleservice

This is a hacky PR to make AWS Batch WorkerManagers upload to s3, instead of the bundle service (since the bundle service might be flaky).

+11 -6

0 comment

3 changed files

pr created time in 9 days

create barnchnelson-liu/codalab-worksheets

branch : upload_to_aws

created branch time in 9 days

PR opened conda-forge/staged-recipes

Add recipe for asset

<!-- Thank you very much for putting in this recipe PR!

This repository is very active, so if you need help with a PR or once it's ready for review, please let the right people know. There are language-specific teams for reviewing recipes.

Currently available teams are:

  • python @conda-forge/help-python
  • python/c hybrid @conda-forge/help-python-c
  • r @conda-forge/help-r
  • java @conda-forge/help-java
  • nodejs @conda-forge/help-nodejs
  • c/c++ @conda-forge/help-c-cpp
  • perl @conda-forge/help-perl
  • Julia @conda-forge/help-julia
  • ruby @conda-forge/help-ruby

If your PR doesn't fall into those categories please contact the full review team @conda-forge/staged-recipes.

Due to GitHub limitations first time contributors to conda-forge are unable to ping these teams. You can ping the team using a special command in a comment on the PR to get the attention of the staged-recipes team. You can also consider asking on our Gitter channel or on our Keybase chat if your recipe isn't reviewed promptly. -->

Checklist

  • [ ] Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml"
  • [ ] License file is packaged (see here for an example)
  • [ ] Source is from official source
  • [ ] Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged)
  • [ ] If static libraries are linked in, the license of the static library is packaged.
  • [ ] Build number is 0
  • [ ] A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details)
  • [ ] GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there
+44 -0

0 comment

1 changed file

pr created time in 9 days

create barnchnelson-liu/staged-recipes

branch : asset

created branch time in 9 days

issue openedcodalab/codalab-worksheets

Worker does not exist when BundleManager tries to remove it

I'm occasionally seeing this in the BundleManager logs:

2020-10-12 17:55:34,709 Cleaning up dead worker (0xdea2e2ecd0084134bacfdec87cff1186, tonyhlee-codalab-slurm-worker-stanford-standard-4670c69e)
Traceback (most recent call last):
  File "/opt/codalab-worksheets/codalab/server/bundle_manager.py", line 83, in run
    self._run_iteration()
  File "/opt/codalab-worksheets/codalab/server/bundle_manager.py", line 103, in _run_iteration
    self._schedule_run_bundles()
  File "/opt/codalab-worksheets/codalab/server/bundle_manager.py", line 782, in _schedule_run_bundles
    self._schedule_run_bundles_on_workers(workers, staged_bundles_to_run, user_info_cache)
  File "/opt/codalab-worksheets/codalab/server/bundle_manager.py", line 424, in _schedule_run_bundles_on_workers
    self._cleanup_dead_workers(workers)
  File "/opt/codalab-worksheets/codalab/server/bundle_manager.py", line 267, in _cleanup_dead_workers
    workers.remove(worker['worker_id'])
  File "/opt/codalab-worksheets/codalab/server/worker_info_accessor.py", line 20, in wrapper
    return f(*args, **kwargs)
  File "/opt/codalab-worksheets/codalab/server/worker_info_accessor.py", line 75, in remove
    worker = self._workers[worker_id]
KeyError: 'tonyhlee-codalab-slurm-worker-stanford-standard-4670c69e'

I think that this could prevent the bundlemanager from scheduling new bundles, so we should make this more robust.

created time in 9 days

Pull request review commentconda-forge/staged-recipes

Add recipe for aadict

+{% set name = "aadict" %}+{% set version = "0.2.3" %}+++package:+  name: {{ name|lower }}+  version: {{ version }}++source:+  url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/aadict-{{ version }}.tar.gz+  sha256: a77328ac55dbb5735da99441870251befe135f687ab707a7a178561363b27704++build:+  number: 0+  noarch: python+  script: "{{ PYTHON }} -m pip install . --no-deps -vv"++requirements:+  host:+    - pip+    - python+  run:+    - python+    - six >=1.6.0++test:+  imports:+    - aadict+  commands:+    - pip check+  requires:+    - pip++about:+  home: http://github.com/metagriffin/aadict+  summary: An auto-attribute dict (and a couple of other useful dict functions)+  license: GPL-2.0-or-later

thanks, fixed!

nelson-liu

comment created time in 9 days

PullRequestReviewEvent

push eventnelson-liu/staged-recipes

Nelson Liu

commit sha 9c97e5ba506b46ddf6ff082305d4a14a0d319dcb

Update recipes/aadict/meta.yaml

view details

push time in 9 days

Pull request review commentconda-forge/staged-recipes

Add recipe for aadict

+{% set name = "aadict" %}+{% set version = "0.2.3" %}+++package:+  name: {{ name|lower }}+  version: {{ version }}++source:+  url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/aadict-{{ version }}.tar.gz+  sha256: a77328ac55dbb5735da99441870251befe135f687ab707a7a178561363b27704++build:+  number: 0+  noarch: python+  script: "{{ PYTHON }} -m pip install . --no-deps -vv"++requirements:+  host:+    - pip+    - python+  run:+    - python+    - six >=1.6.0++test:+  imports:+    - aadict+  commands:+    - pip check+  requires:+    - pip++about:+  home: http://github.com/metagriffin/aadict+  summary: An auto-attribute dict (and a couple of other useful dict functions)+  license: GPL-2.0-or-later
  license: GPL-3.0-or-later
nelson-liu

comment created time in 9 days

PullRequestReviewEvent

issue openedhuggingface/transformers

SQuAD example docs inaccurately suggest settings for bert-large-uncased on a single V100

Hi!

In the SQuAD example docs, it's noted that ( https://github.com/huggingface/transformers/blob/3f42eb979f7bd20448ff6b15ab316d63f5489a6f/docs/source/examples.md#fine-tuning-bert-on-squad10 ) This example code fine-tunes BERT on the SQuAD1.0 dataset. It runs in 24 min (with BERT-base) or 68 min (with BERT-large) on a single tesla V100 16GB.

Could just be me, but I don't think the example as provided works with s/bert-base-uncased/bert-large-uncased/---even with FP16, you run into a GPU OOM.

It might be useful to revisit this recommendation and/or remove it.

created time in 10 days

issue openedcodalab/codalab-worksheets

Hostname does not render properly in remote field on AWS Batch WorkerManager-submitted jobs

Describe the bug

This is a bug with https://github.com/codalab/codalab-worksheets/pull/2894 . In that PR, we modified the WorkerManager's launch command to add the node's $(hostname -s) to the worker ID.

However, this doesn't work for AWS batch---I assume that they have something that escapes out these shell commands in the "command" field of batch jobs? What actually shows up in the remote is simply: $(hostname -s)-1f765cef467b478d9ec1a12b7cc1eb33

created time in 10 days

push eventnelson-liu/codalab-worksheets

yuqijin

commit sha 04fff6e34c00c2de79bf66b97cc450d617eb9741

Bump to version 0.5.23

view details

Tony Lee

commit sha 830596f16522ef84dec9a9a1539a06d5ba218832

fix --job-filter (#2945)

view details

Ashwin Ramaswami

commit sha 9d286ec7f81c04e0a127884c5c0d023093c9f364

BundleManager tests (#2795) * reorganize tests, add webtest for rest api tests * Add frontend tests * Update some documentation * fix unit test run command * add rest api tests * update test * fix test files location * black format * fix tests * black format * refactor, fix rest api tests * add testing philosophy * Update snapshots * Add code coverage checks * update nomenclature * fix unittest * cleanup * fix path * fix test path * fix test * Fix tests, make running unit tests easier * try test fixes * add chown command * fix coverage * fix test * fix test again * fix test * fix test * fix * add sqlite model * Add bundle manager tests * fix sqlite db to work across threads * more tests, fix issue where cpus defaults to 1 * Add BundleManagerFailUnresponsiveBundlesTest * more tests * more tests * split test into multiple files * Update codalab/server/bundle_manager.py * fix tests * undo * more abstractions * refactor * freezegun * refactor * add comments * update names * format * typing * Fix typing * refactor * more refactoring * Add comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Ashwin Ramaswami

commit sha 5c4b76bc594c0f26176fa82b3ea27cf2a845d79d

Run github actions on all PRs, even those that are not to master (#2956)

view details

Ashwin Ramaswami

commit sha 414c2c3d186baf3f43ff87f166e86c59b8f222d3

Merge pull request #2932 from codalab/rc0.5.23 Bump to version 0.5.23

view details

Nelson Liu

commit sha 38e5483a54976d2e871280cb3e5013fd01d5cc85

Add CodalabManagerState classes, with support for json and sqlite3

view details

push time in 10 days

push eventnelson-liu/codalab-worksheets

yuqijin

commit sha 04fff6e34c00c2de79bf66b97cc450d617eb9741

Bump to version 0.5.23

view details

Tony Lee

commit sha 830596f16522ef84dec9a9a1539a06d5ba218832

fix --job-filter (#2945)

view details

Ashwin Ramaswami

commit sha 9d286ec7f81c04e0a127884c5c0d023093c9f364

BundleManager tests (#2795) * reorganize tests, add webtest for rest api tests * Add frontend tests * Update some documentation * fix unit test run command * add rest api tests * update test * fix test files location * black format * fix tests * black format * refactor, fix rest api tests * add testing philosophy * Update snapshots * Add code coverage checks * update nomenclature * fix unittest * cleanup * fix path * fix test path * fix test * Fix tests, make running unit tests easier * try test fixes * add chown command * fix coverage * fix test * fix test again * fix test * fix test * fix * add sqlite model * Add bundle manager tests * fix sqlite db to work across threads * more tests, fix issue where cpus defaults to 1 * Add BundleManagerFailUnresponsiveBundlesTest * more tests * more tests * split test into multiple files * Update codalab/server/bundle_manager.py * fix tests * undo * more abstractions * refactor * freezegun * refactor * add comments * update names * format * typing * Fix typing * refactor * more refactoring * Add comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Ashwin Ramaswami

commit sha 5c4b76bc594c0f26176fa82b3ea27cf2a845d79d

Run github actions on all PRs, even those that are not to master (#2956)

view details

Ashwin Ramaswami

commit sha 414c2c3d186baf3f43ff87f166e86c59b8f222d3

Merge pull request #2932 from codalab/rc0.5.23 Bump to version 0.5.23

view details

Nelson Liu

commit sha f02d36f22adfa5102351768a3c672864144ba232

Add CodalabManagerState classes, with support for json and sqlite3

view details

push time in 10 days

push eventnelson-liu/codalab-worksheets

yuqijin

commit sha 04fff6e34c00c2de79bf66b97cc450d617eb9741

Bump to version 0.5.23

view details

Tony Lee

commit sha 830596f16522ef84dec9a9a1539a06d5ba218832

fix --job-filter (#2945)

view details

Ashwin Ramaswami

commit sha 9d286ec7f81c04e0a127884c5c0d023093c9f364

BundleManager tests (#2795) * reorganize tests, add webtest for rest api tests * Add frontend tests * Update some documentation * fix unit test run command * add rest api tests * update test * fix test files location * black format * fix tests * black format * refactor, fix rest api tests * add testing philosophy * Update snapshots * Add code coverage checks * update nomenclature * fix unittest * cleanup * fix path * fix test path * fix test * Fix tests, make running unit tests easier * try test fixes * add chown command * fix coverage * fix test * fix test again * fix test * fix test * fix * add sqlite model * Add bundle manager tests * fix sqlite db to work across threads * more tests, fix issue where cpus defaults to 1 * Add BundleManagerFailUnresponsiveBundlesTest * more tests * more tests * split test into multiple files * Update codalab/server/bundle_manager.py * fix tests * undo * more abstractions * refactor * freezegun * refactor * add comments * update names * format * typing * Fix typing * refactor * more refactoring * Add comments Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

view details

Ashwin Ramaswami

commit sha 5c4b76bc594c0f26176fa82b3ea27cf2a845d79d

Run github actions on all PRs, even those that are not to master (#2956)

view details

Ashwin Ramaswami

commit sha 414c2c3d186baf3f43ff87f166e86c59b8f222d3

Merge pull request #2932 from codalab/rc0.5.23 Bump to version 0.5.23

view details

push time in 10 days

push eventconda-forge/jsonnet-feedstock

regro-cf-autotick-bot

commit sha 8fdfbba4fa5232589edeef116d26367dec7743ff

Rebuild for python39

view details

regro-cf-autotick-bot

commit sha 1dd7401fd7ba5b68872f074a5e4a383e8b553311

MNT: Re-rendered with conda-build 3.20.3, conda-smithy 3.8.1, and conda-forge-pinning 2020.10.09.16.47.17

view details

Nelson Liu

commit sha 789c7965bd83f20536c3fe3584664b9bd5c8174c

Merge pull request #18 from regro-cf-autotick-bot/rebuild-python39-0-2_h9539c9 Rebuild for python39

view details

push time in 10 days

PR merged conda-forge/jsonnet-feedstock

Reviewers
Rebuild for python39

This PR has been triggered in an effort to update python39.

Notes and instructions for merging this PR:

  1. Please merge the PR only after the tests have passed.
  2. Feel free to push to the bot's branch to update this PR if needed.

Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.

This package has the following downstream children:

And potentially more.

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase <code>@<space/>conda-forge-admin, please rerun bot</code> in a PR comment to have the conda-forge-admin add it for you.

<sub>This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. If you would like a local version of this bot, you might consider using rever. Rever is a tool for automating software releases and forms the backbone of the bot's conda-forge PRing capability. Rever is both conda (conda install -c conda-forge rever) and pip (pip install re-ver) installable. Finally, feel free to drop us a line if there are any issues! This PR was generated by , please use this URL for debugging</sub>

+408 -160

1 comment

33 changed files

regro-cf-autotick-bot

pr closed time in 10 days

create barnchnelson-liu/staged-recipes

branch : aadict_recipe

created branch time in 11 days

pull request commentconda-forge/staged-recipes

Add recipe for globre

@conda-forge/help-python this should be ready for a look

nelson-liu

comment created time in 11 days

pull request commentconda-forge/staged-recipes

Add recipe for aadict

@conda-forge/help-python this should be ready for a look

nelson-liu

comment created time in 11 days

more