profile
viewpoint

isaprykin/sf-bay-area-hikes 1

Good places to hike or walk outside in SF Bay area

isaprykin/mario-rl 0

Implementation of a DQN for Mario that I started on a plane ride once.

isaprykin/PyGrid 0

A Peer-to-peer Platform for Secure, Privacy-preserving, Decentralized Data Science

isaprykin/transformers-sota 0

Simple from-scratch implementations of transformer-based models that match the state of the art.

issue openedgolemfactory/golem

Officially support Debian : I hacked the script and it's easy.

You can easily support Debian as well.

As of today, if I run ./install.sh on Debian then I get

ERROR: Cannot check package installed. Distribution not supported: debian. No such function: check_package_installed_version_debian.
ERROR: Cannot install nvidia-docker. Distribution not supported: debian. No such function: install_nvidia_docker_debian.

If I hack up the script as follows:

$ diff original_install.sh debian_install.sh 
51c51
< DOCKER_GPG_KEY="https://download.docker.com/linux/ubuntu/gpg"
---
> DOCKER_GPG_KEY="https://download.docker.com/linux/debian/gpg"
258,259c258,260
<   local distro_name=$( lsb_release -is 2>/dev/null | awk '{print tolower($0)}' )
<   echo $distro_name
---
>   # local distro_name=$( lsb_release -is 2>/dev/null | awk '{print tolower($0)}' )
>   # echo $distro_name
>   echo "ubuntu"
392c393
<   $( sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" > /dev/null ) &
---
>   $( sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" > /dev/null ) &

and apt install nvidia-docker-container then the installation script works as follows.

Support this officially to enable more users.

created time in 13 days

startedUniswap/contracts-vyper

started time in 18 days

startedmakerdao/simple-arbitrage-keeper

started time in 20 days

startedmakerdao/awesome-makerdao

started time in 21 days

startedbrndnmtthws/optimal-buy-cbpro

started time in a month

startedid-Software/DOOM

started time in a month

startedSkeltonThatcher/run-book-template

started time in a month

starteddjcb/mu

started time in a month

issue commentkaiwk/leetcode.el

Add to Melpa stable

It appears so! https://stable.melpa.org/#/leetcode Thank you, how awesome :-)

isaprykin

comment created time in a month

startedmattermost/mattermost-server

started time in a month

issue openedkaiwk/leetcode.el

Add to Melpa stable

Hello! This package looks great.

Could you please create a stable version and add it to Melpa stable?

I'm trying to maximize the stability of my emacs and I'm trying to use Melpa stable, so that the installed packages are not just HEAD version of github repos. Thank you!

It seems like you just need to create a git tag for versions.

created time in a month

startedbitwarden/server

started time in a month

startedfogleman/primitive

started time in a month

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha ba172d27e4e0881207dc75ea00365e64f6916e88

Flake8 should allow two space identation for code comments. It is already allowed by means of ignoring E111. E114 is the corresponding error for code comments.

view details

Igor Saprykin

commit sha 973852a79af39029ad86f6c5277d7dda0bf53d68

Avoids re-generating batches if they are found on disk. In order to improve iteration speed, batches are not going to be renegerated every run. The input data will likely be the same between runs unless the 'batches file' is removed from disk.

view details

push time in a month

startedakroshko/emacs-otdb

started time in a month

startednextcloud/desktop

started time in a month

startedlepisma/rogue

started time in a month

issue closedemacs-pe/docker-tramp.el

Works with some docker images, but doesn't work with nvcr.io/nvidia/tensorflow:19.12-tf1-py3

Hello! This utility would have been very convenient if it worked for me.

I can connect to a running instance of nvidia/cuda, but not nvcr.io/nvidia/tensorflow:19.12-tf1-py3. Could you please help me? I tried running with tramp-verbose 6, but I can't figure out what's so different about the two scenarios when it works and when it doesn't.

Tramp: Opening connection for cedf99ccbf03 using docker...
Tramp: Sending command ‘exec docker  exec -it  cedf99ccbf03 sh’
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on ‘cedf99ccbf03’
Tramp: Opening connection for cedf99ccbf03 using docker...failed

This is GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-09-22, modified by Debian

closed time in a month

isaprykin

startedsecuras/SealedBite

started time in 2 months

startedpanicsteve/cloud-to-butt

started time in 2 months

startedhuggingface/tflite-android-transformers

started time in 2 months

startedpytorch/fairseq

started time in 2 months

issue commentemacs-pe/docker-tramp.el

Works with some docker images, but doesn't work with nvcr.io/nvidia/tensorflow:19.12-tf1-py3

Okay, phew! It's still sad, but I after I symlinked sh into bash inside the container I was able to connect.

isaprykin

comment created time in 2 months

issue openedemacs-pe/docker-tramp.el

Works with some docker images, but doesn't work with nvcr.io/nvidia/tensorflow:19.12-tf1-py3

Hello! This utility would have been very convenient if it worked for me.

I can connect to a running instance of nvidia/cuda, but not nvcr.io/nvidia/tensorflow:19.12-tf1-py3. Could you please help me? I tried running with tramp-verbose 6, but I can't figure out what's so different about the two scenarios when it works and when it doesn't.

Tramp: Opening connection for cedf99ccbf03 using docker...
Tramp: Sending command ‘exec docker  exec -it  cedf99ccbf03 sh’
Tramp: Waiting for prompts from remote shell...done
Tramp: Found remote shell prompt on ‘cedf99ccbf03’
Tramp: Opening connection for cedf99ccbf03 using docker...failed

"This is GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2019-09-22, modified by Debian"

created time in 2 months

startedJAremko/docker-x11-bridge

started time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha c78cd7291597890d391e2b9a56dadd16d06c5557

Configure flake8 for code editors. The initial configuration makes flake8 agree with the two-space identation and the line length of 80.

view details

push time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha e35448924d54c8692b392ae74f8242a53bc5a846

Fix an issue with empty batches for sequences of max_length. If the sequence is max_length, then it was in max_length+1 max boundary bucket. Therefore max_length // max_length + 1 is 0 and 0 sequences were put in the batch. The fix attempts to put 1 sequence of max_length into such a batch.

view details

Igor Saprykin

commit sha 0adb21dcf559dc96d98d90191cc7b575de3d1ecb

Use torch.split instead of torch.chunk for correct batch sizes. torch.chunk(t, 5) splits t into 5 parts, whereas what I need is torch.split(t, 5) that is going to split t into parts 5 elements each. This allows for correct batch sizes.

view details

push time in 2 months

startedbastibe/org-journal

started time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha abf5e951d0e52167e73762e224b67df100a1e87c

Disregard sentences that are longer than the batch size. This treatment is consistent with the current packing mechanism where we try to fill the batch up to a particular size with a variable number of sequences.

view details

push time in 2 months

startedreal-logic/aeron

started time in 2 months

startedreal-logic/simple-binary-encoding

started time in 2 months

startedLMAX-Exchange/disruptor

started time in 2 months

startedtroglobit/inadyn

started time in 2 months

issue commentNVIDIA/nvidia-docker

Debian 10 (Buster) error response from daemon: Unknown runtime specified nvidia. / OCI runtime create failed.

I struggled with this problem on the same setup with Debian 10 too. The minimal solution turned out to be to install the libcuda1 package.

svdHero

comment created time in 2 months

startedvtl/ethblk

started time in 2 months

startedjheusser/satcoin

started time in 2 months

startedcolah/ImplicitCAD

started time in 2 months

startedcanadaduane/vscode-notes

started time in 2 months

starteddeepset-ai/bert-tensorflow

started time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha 250c419c9593fc72714b5051a48372e3536beac7

Remove unnecessary parameter. There is no need to pass self._dim to PatBatch._pad_tensor because that method has access to it anyway.

view details

push time in 2 months

PR opened OpenMined/PyGrid

Correct Fell->Feel type in the README.

There is a noticeable typo. Please accept this patch to fix it.

Pull Request Template

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please mark options that are relevant.

  • [ ] Bug fix (non-breaking change which fixes an issue)
  • [ ] New feature (non-breaking change which adds functionality)
  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • [ ] This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • [ ] Test A
  • [ ] Test B

Checklist:

  • [ ] I did follow the contribution guidelines
  • [ ] New Unit tests added
  • [ ] Unit tests pass locally with my changes
+1 -1

0 comment

1 changed file

pr created time in 2 months

push eventisaprykin/PyGrid

Igor

commit sha 831c12b4b06e909053fdf369a69ac1f16f38988e

Correct Fell->Feel type in the README. There is a noticeable typo. Please accept this patch to fix it.

view details

push time in 2 months

fork isaprykin/PyGrid

A Peer-to-peer Platform for Secure, Privacy-preserving, Decentralized Data Science

fork in 2 months

issue openedflorian/federated-learning

A question on "Federated Learning for Firefox".

I got curious about one point when reading https://florian.github.io/federated-learning-firefox/.

It is something that is described in the paragraph that starts with "From a user perspective, it is not clear if these changes improve the user experience."

Would it have been informative to compute "mean selected rank" for the subset of the treatment group that typed longer sequences such that the "mean characters typed" of the subset is around 4.25?

Thanks. P.S. I didn't find a better forum to ask this question.

created time in 2 months

startedcybertronai/pytorch-sso

started time in 2 months

startedOpenMined/PyGrid

started time in 2 months

startedOpenMined/PySyft

started time in 2 months

startedunconst/BitTensor

started time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha 66c338c5cc36817e1a722d1ae3dea0210a0fed45

Batch up to batch_size when max length is unspecified.

view details

push time in 2 months

push eventisaprykin/transformers-sota

push time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha 6488303e7d93d269db1f2f13acf0eda269916e6b

Batch up to batch_size when maximum_length is None.

view details

Igor Saprykin

commit sha 0a4391c2016ad364439d22523d5bd30170bd38cc

Merge branch 'master' of github.com:isaprykin/transformers-sota

view details

push time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha befb58b42b575574739801a7e093fc06a902e202

Batch up to batch_size when maximum_length is None.

view details

Igor Saprykin

commit sha 91718936c5dbc92988af18736ec739381c581833

Merge branch 'master' of github.com:isaprykin/transformers-sota

view details

push time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha 02cc739ec42760dbb4461c09f2724d6bd49c2888

Batch up to batch_size when maximum_length is None.

view details

push time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha bdda55c2222d7460d4274f0ccce4af7a4d5b2e0f

Batch the encoded sequences. The sequences are variable-length, so the goal of batching is to maintain the constant total length in the batch. A batch might have a lot of small sequences or a small amount of large sequences.

view details

Igor Saprykin

commit sha 9a16ba3e4dc2c2fe8c9f91ec1d848902ff3e0dab

Add .gitignore file. Update .pylintrc It doesn't seem right to version control a few file types. Pylintrc would report wrong errors on the "dynamically-linked" torch modules.

view details

push time in 2 months

issue commenttensorflow/tensor2tensor

Multi-GPU gives no speedup for transformer model

@lkluo Well, what I can say, based on observations, is that 4 GPUs let the model converge faster and overall better. I guess it is due to a larger batch size. From my experience I'd recommend using multiple GPUs and to use larger batch sizes.

How long does did it take you to reach SOTA on 4 GPUs?

stefan-falk

comment created time in 2 months

startedmillejoh/emacs-ipython-notebook

started time in 2 months

startedanoidgit/transformer

started time in 2 months

startedjorgenschaefer/elpy

started time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha 48b66b99078fbf4889a05b826b7a13860ba3f970

Add requirements.txt that summarizes the dependencies. This file might be useful to recreate the working virtual environment. It could be done via `$ pip install -r requirements.txt`.

view details

Igor Saprykin

commit sha d6b3255e83ccd539903feccfb0e110e0aad75f19

Encode the whole corpus and store it locally. The model could read these files, shuffle and batch them and then use the data for training.

view details

Igor Saprykin

commit sha 697fd7e5a3aea7bbc43300effd7157d8ee42fb96

Add eval dataset and split them into trainining and eval datasets. Vocabulary is going to be learned only on the training dataset.

view details

push time in 2 months

startedjwiegley/use-package

started time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha b89e63d88443afd32959427c126921b23d50b0d5

Re-store the vocabulary from a file if possible. This is can happen as an alternative to re-building the vocabulary from scratch. I added a simple unit test as well. This commit also contains formatting changes.

view details

push time in 2 months

startedmagit/magit

started time in 2 months

startedmagit/forge

started time in 2 months

push eventisaprykin/transformers-sota

Igor Saprykin

commit sha 88bc87404eade71fa60efb8cd1e94ecfebce63c8

Fix lint errors in main.py. This makes the whole current codebase lint free.

view details

push time in 2 months

push eventisaprykin/transformers-sota

Igor

commit sha 52586e38a95df2142fd4bc9811d2174314939c70

Add a LICENSE. MIT License is allows for the maximum collaboration.

view details

push time in 2 months

create barnchisaprykin/transformers-sota

branch : master

created branch time in 2 months

created repositoryisaprykin/transformers-sota

Simple implementations of transformer-based models from scratch that match the state of the art.

created time in 2 months

create barnchisaprykin/transformers-sota

branch : master

created branch time in 2 months

created repositoryisaprykin/transformers-sota

Simple implementations of transformer-based models from scratch that match the state of the art.

created time in 2 months

startedremacs/remacs

started time in 2 months

startedjonathanmetzman/wasm-fuzzing-demo

started time in 2 months

startedbbatsov/projectile

started time in 2 months

startedkoekeishiya/yabai

started time in 2 months

startedRadeonOpenCompute/ROCm

started time in 2 months

startedRadeonOpenCompute/hcc

started time in 2 months

issue closedhuggingface/transformers

Wrong paraphrase in the TF2/PyTorch README example.

🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): TFBertForSequenceClassification

Language I am using the model on (English, Chinese....): English

The problem arise when using:

  • [x] the official example scripts: https://github.com/huggingface/transformers#quick-tour-tf-20-training-and-pytorch-interoperability
  • [ ] my own modified scripts: (give details)

The tasks I am working on is:

  • [x] an official GLUE/SQUaD task: Sequence Classification
  • [ ] my own task or dataset: (give details)

To Reproduce

Steps to reproduce the behavior:

  1. Run the attached script.
  2. Observe
$ /Users/igor/projects/ml-venv/bin/python /Users/igor/projects/transformers-experiments/paraphrasing_issue.py
2019-11-25 08:58:53.985213: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fed57a2be00 executing computations on platform Host. Devices:
2019-11-25 08:58:53.985243: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
INFO:absl:Overwrite dataset info from restored data version.
INFO:absl:Reusing dataset glue (/Users/igor/tensorflow_datasets/glue/mrpc/0.0.2)
INFO:absl:Constructing tf.data.Dataset for split None, from /Users/igor/tensorflow_datasets/glue/mrpc/0.0.2
Train for 115 steps, validate for 7 steps
Epoch 1/2
  4/115 [>.............................] - ETA: 1:22:04 - loss: 0.6936  5/115 [>.............................] - ETA: 1:18:44 - loss: 0.6876  6/115 [>.............................] - ETA: 1:16:01 - loss: 0.6760115/115 [==============================] - 4587s 40s/step - loss: 0.5850 - accuracy: 0.7045 - val_loss: 0.4695 - val_accuracy: 0.8137
Epoch 2/2
115/115 [==============================] - 4927s 43s/step - loss: 0.3713 - accuracy: 0.8435 - val_loss: 0.3825 - val_accuracy: 0.8358
**sentence_1 is a paraphrase of sentence_0
sentence_2 is a paraphrase of sentence_0**
  1. Wonder why.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

import tensorflow as tf
import tensorflow_datasets
from transformers import *

# Load dataset, tokenizer, model from pretrained model/vocabulary
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
model = TFBertForSequenceClassification.from_pretrained('bert-base-cased')
data = tensorflow_datasets.load('glue/mrpc')

# Prepare dataset for GLUE as a tf.data.Dataset instance
train_dataset = glue_convert_examples_to_features(data['train'], tokenizer, max_length=128, task='mrpc')
valid_dataset = glue_convert_examples_to_features(data['validation'], tokenizer, max_length=128, task='mrpc')
train_dataset = train_dataset.shuffle(100).batch(32).repeat(2)
valid_dataset = valid_dataset.batch(64)

# Prepare training: Compile tf.keras model with optimizer, loss and learning rate schedule 
optimizer = tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metric = tf.keras.metrics.SparseCategoricalAccuracy('accuracy')
model.compile(optimizer=optimizer, loss=loss, metrics=[metric])

# Train and evaluate using tf.keras.Model.fit()
history = model.fit(train_dataset, epochs=2, steps_per_epoch=115,
                    validation_data=valid_dataset, validation_steps=7)

# Load the TensorFlow model in PyTorch for inspection
model.save_pretrained('./save/')
pytorch_model = BertForSequenceClassification.from_pretrained('./save/', from_tf=True)

# Quickly test a few predictions - MRPC is a paraphrasing task, let's see if our model learned the task
sentence_0 = "This research was consistent with his findings."
sentence_1 = "His findings were compatible with this research."
sentence_2 = "His findings were not compatible with this research."
inputs_1 = tokenizer.encode_plus(sentence_0, sentence_1, add_special_tokens=True, return_tensors='pt')
inputs_2 = tokenizer.encode_plus(sentence_0, sentence_2, add_special_tokens=True, return_tensors='pt')

pred_1 = pytorch_model(inputs_1['input_ids'], token_type_ids=inputs_1['token_type_ids'])[0].argmax().item()
pred_2 = pytorch_model(inputs_2['input_ids'], token_type_ids=inputs_2['token_type_ids'])[0].argmax().item()

print("sentence_1 is", "a paraphrase" if pred_1 else "not a paraphrase", "of sentence_0")
print("sentence_2 is", "a paraphrase" if pred_2 else "not a paraphrase", "of sentence_0")

Expected behavior

sentence_1 is a paraphrase of sentence_0
sentence_2 is not a paraphrase of sentence_0

Environment

  • OS: MacOS
  • Python version: 3.7.5
  • PyTorch version: 1.3.1
  • PyTorch Transformers version (or branch): last commit afaa33585109550f9ecaaee4e47f187aaaefedd0 as of Sat Nov 23 11:34:45 2019 -0500.
  • Using GPU ? nope
  • Distributed of parallel setup ? single machine
  • Any other relevant information: TF version is 2.0.0

closed time in 2 months

isaprykin

issue commenthuggingface/transformers

Wrong paraphrase in the TF2/PyTorch README example.

Sounds like we don't think there's an actionable issue here.

isaprykin

comment created time in 2 months

issue closedarxiv-vanity/arxiv-vanity

Failed to render BERT paper

Why shouldn't it render?

" The paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" failed to render. Take a look at the original PDF instead.

Go back home"

closed time in 2 months

isaprykin

startedmoses-smt/mosesdecoder

started time in 3 months

startedNVIDIA/DeepLearningExamples

started time in 3 months

startedkubeflow/kubeflow

started time in 3 months

startedlightvector/KataGo

started time in 3 months

startedopenai/sparse_attention

started time in 3 months

startedthomwolf/Magic-Sand

started time in 3 months

startedopenai/gpt-2

started time in 3 months

startedgoogle/sentencepiece

started time in 3 months

startedandreamad8/Universal-Transformer-Pytorch

started time in 3 months

issue openedarxiv-vanity/arxiv-vanity

Failed to render BERT paper

Why shouldn't it render?

" The paper "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" failed to render. Take a look at the original PDF instead.

Go back home"

created time in 3 months

startedgoogle/gin-config

started time in 3 months

startedcfiken/universal_transformer

started time in 3 months

startedgoogle-research/text-to-text-transfer-transformer

started time in 3 months

issue commenthuggingface/transformers

Wrong paraphrase in the TF2/PyTorch README example.

Thanks for the investigation. Was the performance ever different at the time when that example was put into the README?

isaprykin

comment created time in 3 months

startedtorch-js/torch-js

started time in 3 months

startedandreasjansson/language-detection.el

started time in 3 months

startedarxiv-vanity/engrafo

started time in 3 months

startedarxiv-vanity/arxiv-vanity

started time in 3 months

startedMostafaDehghani/bAbI-T2T

started time in 3 months

startedpytorch/pytorch

started time in 3 months

startedkpot/keras-transformer

started time in 3 months

startedslavpetrov/berkeleyparser

started time in 3 months

more