profile
viewpoint
Karan Desai kdexd University of Michigan Ann Arbor, MI kdexd.github.io

kdexd/digit-classifier 781

A single handwritten digit classifier, using the MNIST dataset. Pure Numpy.

kdexd/yolog 312

Beautify your Git Logs !

kdexd/virtex 259

Code for the paper "VirTex: Learning Visual Representations from Textual Annotations"

batra-mlp-lab/visdial 212

[CVPR 2017] Torch code for Visual Dialog

batra-mlp-lab/visdial-challenge-starter-pytorch 167

Starter code in PyTorch for the Visual Dialog challenge

kdexd/lang-emerge-parlai 105

Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

kdexd/maxmin-cnn 73

Implementation of Paper: "MaxMin Convolution Neural Networks for Image Classfication" in Keras.

kdexd/probnmn-clevr 52

Code for ICML 2019 paper "Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering" [long-oral]

mdg-iitr/Magneto-Mania 14

A simple game for android, with an addictive gameplay, where you need to dodge the attacks of an angry Monster Ball, and its weapons.

nocaps-org/image-feature-extractors 14

Feature extraction and visualization scripts for nocaps baselines.

issue commentfacebookresearch/swav

TypeError: optimizers must be either a single optimizer or a list of optimizers.

https://github.com/NVIDIA/apex/issues/978 is probably related.

guerriep

comment created time in a day

issue commentfacebookresearch/swav

TypeError: optimizers must be either a single optimizer or a list of optimizers.

+1 facing the same issue, following this thread.

guerriep

comment created time in 3 days

issue commentkdexd/virtex

unable to find a valid cuDNN algorithm to run convolution

Awesome, glad it worked for you!

Charlie-zhang1406

comment created time in 8 days

issue commentkdexd/virtex

unable to find a valid cuDNN algorithm to run convolution

Thanks for trying out the code! I am glad that you at least got it working without automatic mixed-precision, which should be perfectly fine if you have enough GPU memory (it will not significantly affect the results or compatibility).

I could not reproduce this issue unfortunately, I am using CUDA 10.1 and CuDNN 8.0.3 As a sanity check, make sure you got NVIDIA Apex installed properly (with CUDA extensions) as mentioned in its README.

Charlie-zhang1406

comment created time in 9 days

issue closedkdexd/virtex

Reproduction of SPICE results (eval_captioning.py)

Hi, nice work - thanks for sharing the code.

I'm trying to reproduce CIDEr & SPICE results, as appear in figure 4 in your paper. I simply load the pre-trained models (specifically, those that correspond to H=1024 & H=2048 in width-ablation) and run eval_captioning.py, after building the vocabulary. The values I get are much lower than those in fig. 4, which seems like some inconsistency in the pre/post-processing. Should I expect the same values in this experiment? If so, is there any change I should perform?

closed time in 17 days

Eladhi

issue commentkdexd/virtex

Reproduction of SPICE results (eval_captioning.py)

Fixed in 3b6d628 — I can reproduce the results now. Feel free to re-open this if you face any problems @Eladhi!

Eladhi

comment created time in 17 days

push eventkdexd/virtex

Karan Desai

commit sha 3b6d62890534768f5699021b53bc9002f647fdb1

Fix beam search bug introduced by visual projection renaming.

view details

push time in 17 days

issue commentkdexd/virtex

Reproduction of SPICE results (eval_captioning.py)

Hi @Eladhi! Thanks for trying out the code. I can confirm this issue, must have slipped in some recent parameter renaming. I will look into it this weekend and let you know.

Eladhi

comment created time in 20 days

pull request commentbatra-mlp-lab/visdial-challenge-starter-pytorch

Bump tensorflow from 1.12.0 to 1.15.4

It's okay dude, chill out.

dependabot[bot]

comment created time in 25 days

issue closedkdexd/virtex

Running Image Captioning Inference on Arbitrary Images: FileNotFound

python scripts/eval_captioning.py \ --config /tmp/bicaptioning_R_50_L1_H2048/pretrain_config.yaml \ --checkpoint-path /tmp/bicaptioning_R_50_L1_H2048/checkpoint_500000.pth \ --data-root /path/to/images_dir \ --output /path/to/save/predictions.json \ --num-gpus-per-machine 1 \ --cpu-workers 4

Hi, I am wondering where can I download the pretrain_config.yaml and the checkpoint file. I cannot find the link on Github README. Is the only way to get the pre-trained model retraining the whole model?

Cheers,

closed time in a month

xden2331

issue commentkdexd/virtex

Running Image Captioning Inference on Arbitrary Images: FileNotFound

I am assuming that it worked out for you; I am closing this issue due to lack of activity for a month. Feel free to reopen if you have any concerns, thanks!

xden2331

comment created time in a month

push eventkdexd/virtex

Karan Desai

commit sha e794906f3d64563b6031c79cf33b89f80f15a7a0

Add acknowledgments on landing page.

view details

push time in a month

push eventkdexd/virtex

Karan Desai

commit sha 6f14c35bbd01eeedfcb98686bfb868743494e732

Add acknowledgments in README.md

view details

push time in a month

push eventkdexd/virtex

Karan Desai

commit sha a9d0e12d7b08b1ba5b560f64787129fe10c855ad

Add acknowledgments in README.md

view details

push time in a month

push eventkdexd/virtex

Karan Desai

commit sha a5338844327ea8891c90841fac5e2d5391f84975

Add acknowledgements on landing page.

view details

push time in a month

push eventkdexd/virtex

Karan Desai

commit sha 9d75281aeb9982e32bc293146123357f7efefb39

Add acknowledgements in README.md

view details

push time in a month

startedayellapragada/root

started time in a month

startedkevinzakka/torchkit

started time in a month

starteddbolya/tide

started time in 2 months

startedalbumentations-team/albumentations

started time in 2 months

push eventkdexd/virtex

Karan Desai

commit sha 576988a9f5c1aa029b981cbdcefbd1fbfce7e655

Fix file path in setup instructions.

view details

push time in 2 months

push eventkdexd/virtex

Karan Desai

commit sha 21b317b2912dae67800311bac44c3a9a89cf647b

BREAKING: move visual projection layer into textual head.

view details

push time in 2 months

issue openedpytorch/pytorch

Allow custom kwargs for forward method of nn.TransformerEncoderLayer, nn.TransformerDecoderLayer.

🚀 Feature

A simple feature to allow passing custom keyword arguments for nn.TransformerEncoderLayer and nn.TransformerDecoderLayer. Ideally, users should be able to flexibly use custom_encoder and custom_decoder arguments of nn.Transformer which can accept arguments of any arbitrary name.

Motivation

Currently, the forward method of any custom encoders and decoders require to follow the exact API ofnn.TransformerEncoderandnn.TransformerDecoder` — what if custom transformer layers require some extra input tensors? For example, DETR injects sinusoidal positional embedding at every encoder layer. To enable this feature, one needs to reimplement/copy-paste large chunks of the codebase, like DETR does here

Pitch

Let the torch.nn.modules.transformer Transformer modules follow this common API:

class Transformer(nn.Module):
    def __init__(self, *existing_pytorch_args):
        # ... existing method body.

    def forward(self, *existing_args,  **encoder_kwargs, **decoder_kwargs):
        # Separate encoder_kwargs and decoder_kargs by checking "encoder_" and "decoder_"
        # prefix. We can use `inpect.signature()` for this.
        memory = self.encoder(..., **encoder_kwargs)
        output = self.decoder(..., **decoder_kwargs)

Similarly, nn.TransformerEncoder should allow **encoder_layer_kwargs and nn.TransformerDecoderLayer should allow **decoder_layer_kwargs.

created time in 2 months

push eventkdexd/virtex

Karan Desai

commit sha 9fd64bda38caeff0597fd07f8afb1fe40df3fcc8

Do not make DistributedSampler when training with CPU.

view details

push time in 2 months

push eventkdexd/virtex

Karan Desai

commit sha 2cb00e67df1eadc41d7463be8c94dc2d0deb4be3

Fix bug for batch_size > 1 in scripts/preprocess/preprocess_coco.py

view details

Karan Desai

commit sha 6c6c722d12307f6dc76905cc398f9898b8f7e47b

Do not make DistributedSampler when training with CPU.

view details

push time in 2 months

starteduntitled-ai/self_supervised

started time in 2 months

startedkarpathy/minGPT

started time in 2 months

issue commentkdexd/virtex

Question about SentencePiece [SOS] and [EOS] ID.

[PAD] and <unk> are same — we use the same token to right-pad captions and represent out-of-vocabulary tokens, similar to recent image captioning models. It is ID 0 by default. As far as I remember, I always refer its token as <unk>, and the corresponding variable name in code is padding_idx.

nooralahzadeh

comment created time in 2 months

issue commentkdexd/virtex

Running Image Captioning Inference on Arbitrary Images: FileNotFound

Hi, you can download the model either as shown by the code example in Model Zoo, or by download URLs in Model Zoo

The corresponding config files are present in configs directory. Please check our Model Zoo docs and match the config file names with the download URLs.

Let me know if you have further questions, thanks!

xden2331

comment created time in 2 months

issue commentkdexd/virtex

Question about SentencePiece [SOS] and [EOS] ID.

Edited title for others to search easily. :-)

nooralahzadeh

comment created time in 2 months

issue commentkdexd/virtex

question

By default, SentencePieceTrainer assigns ID 1 as <s> and ID 2 as </s> . Check here

I prefer [SOS] and [EOS] in text instead of <s> and </s>, so I passed my custom symbols as --control-symbols. Internally, SentencePieceTrainer reserves ID 0 for <unk> (which cannot be changed as far as I know), and other control symbols are assigned from ID 3 (in presence of default <s> and </s>) or ID 1 (in absence of <s> and </s>).

So I turned off default <s> and </s>, and instead provided [SOS] and [EOS] so they get ID 1 and 2 respectively.

nooralahzadeh

comment created time in 2 months

startedtmbdev/webdataset

started time in 2 months

push eventkdexd/virtex

Karan Desai

commit sha 7a58aa594d05751ecb2560e90b5ce66c9faec217

Fix file path in setup instructions.

view details

push time in 2 months

push eventkdexd/virtex

Karan Desai

commit sha b061ee2f79c23635e7598371fc8d7dab7656c4c8

Add an instruction to install NVIDIA Apex.

view details

push time in 2 months

issue commentkdexd/virtex

Difficulty with evaluation on downstream task of Object detection

You need to install https://github.com/nvidia/apex (instructions in README). Word of caution: detection eval requires 13 GB+ GPU memory.

NikhilKanamarla

comment created time in 2 months

push eventkdexd/virtex

Arjun Majumdar

commit sha bc9046c133dfbc5ee0ea6fb1a967b7d087df6cd4

Fix index bug (#10)

view details

push time in 3 months

pull request commentkdexd/virtex

Fix bug in preprocess_coco.py

Oops, sneaky typo.. thanks a lot for the fix!

arjunmajum

comment created time in 3 months

PR merged kdexd/virtex

Fix bug in preprocess_coco.py

Minor bug fix in preprocessing script.

+1 -1

0 comment

1 changed file

arjunmajum

pr closed time in 3 months

push eventkdexd/virtex

Karan Desai

commit sha bf04132ae043e2f72902f4e65c08f5539b81b615

Parallelize data serialization to LMDB.

view details

push time in 3 months

more