profile
viewpoint
Soumith Chintala soumith Facebook AI Research New York, USA http://soumith.ch /\︿╱\ ____________________________________ \0_ 0 /╱\╱ ______________________________ \▁︹_/

pyro-ppl/pyro 6487

Deep universal probabilistic programming with Python and PyTorch

OpenNMT/OpenNMT 2303

Open Source Neural Machine Translation in Torch (deprecated)

pytorch/serve 1257

Model Serving on PyTorch

dmlc/dlpack 333

RFC for common in-memory tensor structure and operator interface for deep learning system

clementfarabet/torch-ios 198

Torch7 for iOS.

prigoyal/pytorch_memonger 186

Experimental ground for optimizing memory of pytorch models

clementfarabet/graphicsmagick 126

A simple Lua wrapper to graphicsmagick.

clementfarabet/lua---nnx 96

An extension to Torch7's nn package.

clementfarabet/lua---parallel 94

A (simple) parallel computing framework for Lua

issue closedpytorch/pytorch

Can't sign up for Pytorch forums

I want to create a new account of Pytorch forums, but the web always says: " something goes wrong, please try forget passwords link..."

closed time in 5 hours

DengNaichen

issue commentpytorch/pytorch

Can't sign up for Pytorch forums

can you see if it works now. we fixed it. please reopen if it doesn't work

DengNaichen

comment created time in 5 hours

issue commentpytorch/pytorch

Can't sign up for Pytorch forums

cc: @ptrblck

DengNaichen

comment created time in 5 hours

issue commentpytorch/pytorch

PyTorch C++ API as a static lib: how to compile ?

@seemethere can you check on this, maybe we've regressed?

VitaMusic

comment created time in 17 hours

issue closedezyang/ghstack

creating a new commit and making it the first commit in the stack has bugs

A, B, C, D are commits

  • master branch -> A -> B -> C
  • ghstack invoked, and PRs created for A, B, C
  • commit D created, and rebased so that master branch -> D -> A -> B -> C
  • ghstack invoked

in this case, the base branch of A is not moved to be D.

Here's the rage log:

https://gist.github.com/soumith/95c36efdb06cd23d98a9727d20042c51

version ghstack v0.2.1

Tried updating to ghstack v0.3.0 and running ghstack again, but it didn't do anything

closed time in 3 days

soumith

issue commentezyang/ghstack

creating a new commit and making it the first commit in the stack has bugs

false report. ignore.

soumith

comment created time in 3 days

issue commentpytorch/pytorch

torch.nn.functional.one_hot should gracefully skip negative and out-of-range indices

the errors should be the default, but let's say F.one_hot(..., relaxed=True) or something can do the more relaxed behavior

msbaines

comment created time in 3 days

issue openedezyang/ghstack

creating a new commit and making it the first commit in the stack has bugs

A, B, C, D are commits

  • master branch -> A -> B -> C
  • ghstack invoked, and PRs created for A, B, C
  • commit D created, and rebased so that master branch -> D -> A -> B -> C
  • ghstack invoked

in this case, the base branch of A is not moved to be D.

Here's the rage log:

https://gist.github.com/soumith/95c36efdb06cd23d98a9727d20042c51

version ghstack v0.2.1

created time in 3 days

pull request commentpytorch/pytorch

[nvFuser] Latency improvements for pointwise + reduction fusion

@suo sorry i didn't see your github comments. I ended up clicking the land button a while ago, and it landed it in the night.

jjsjann123

comment created time in 3 days

push eventpytorch/builder

guyang3532

commit sha ae2d90c920f870363083d15e534f85e0f5f29415

Update Python 3.7 to 3.7.9 (#529)

view details

push time in 4 days

PR merged pytorch/builder

Update Python 3.7 to 3.7.9
+3 -3

2 comments

1 changed file

guyang3532

pr closed time in 4 days

pull request commentpytorch/pytorch

[nvFuser] Latency improvements for pointwise + reduction fusion

got a critical lint issue, possibly from already committed code

This file should only include printable US-ASCII bytes.
Use Unicode escapes like `\u` to represent disallowed values.

torch/csrc/jit/codegen/cuda/lower_unroll.h
jjsjann123

comment created time in 4 days

PullRequestReviewEvent

issue commentpyOpenSci/software-review

pystiche: A Framework for Neural Style Transfer

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

  • [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

  • [x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
  • [x] Installation instructions: for the development version of package and any non-standard dependencies in README
  • [x] Vignette(s) demonstrating major functionality that runs successfully locally
  • [?] Function Documentation: for all user-facing functions
    • not all but ~90% -- see review comments below
  • [ ] Examples for all user-facing functions
    • I didn't see inline doc examples for almost any function. Is this the expectation of this review line item?
  • [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
  • [x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

  • [x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

  • [x] The package name
  • [x] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
  • [x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
  • [x] Installation instructions
  • [x] Any additional setup required (authentication tokens, etc)
  • [ ] Brief demonstration usage
  • [x] Direction to more detailed documentation (e.g. your documentation files or website).
  • [x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
  • [ ] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

  • [x] The documentation is easy to find and understand
  • [x] The need for the package is clear
  • [?] All functions have documentation and associated examples for use
    • ~90% functions have documentation, and the repository examples cover a lot of beginner to complex cases, so I consider this still sufficient

Functionality

  • [x] Installation: Installation succeeds as documented.
  • [ ] Functionality: Any functional claims of the software been confirmed.
    • see my notes below, I had trouble running the example out of the box
  • [x] Performance: Any performance claims of the software been confirmed.
    • no performance claims were made by software
  • [?] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
    • see notes below, had trouble running tests out of the box using instructions. I had to manually install the right packages to get tests working.
  • [x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
  • [?] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.
    • the link to the packaging guidelines is broken

For packages co-submitting to JOSS

  • [x] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

  • [x] A short summary describing the high-level functionality of the software
  • [x] Authors: A list of authors with their affiliations
  • [x] A statement of need clearly stating problems the software is designed to solve and its target audience.
  • [ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).
    • the references section is empty, that doesn't look right: https://github.com/pmeier/pystiche/blob/v0.5.1/paper.md

Final approval (post-review)

  • [ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1.5


Review Comments

The package makes the whole suite of neural style transfer work much easier to comprehend and reliably run. The documentation was nice, the examples were a breeze to read through. There's a lot of value in such a package and it's obvious that a lot of effort and tasteful design have been put into it's development.

Bigger points:

  • The overall API documentation as well as examples are in really good shape. I don't have any real comments here. In-line examples for some of the functional operators (in pystiche.ops and pystiche.ops.functional) would have been nice.
  • I reviewed v0.5.1 (latest stable release) and there seem to be teething problems with the environment I setup (a fresh conda env with python 3.6, and then following package instructions). I've documented it in detail enough that the author should be able to resolve this quickly.
    • Unfortunately, I wasn't able to run the examples locally due to an image hash issue that I've documented below. I also had some issues with running unit tests locally, again documented below. w.r.t the unit tests, I think the tox environment isn't correctly configured. Additionally, some unit tests failed.
install issue

To pip install the package, I tried what was listed in the README, but I got this error:

pip install https://github.com/pmeier/pystiche@v0.5.1
Collecting https://github.com/pmeier/pystiche@v0.5.1
  ERROR: HTTP error 404 while getting https://github.com/pmeier/pystiche@v0.5.1
ERROR: Could not install requirement https://github.com/pmeier/pystiche@v0.5.1 because of HTTP error 404 Client Error: Not Found for url: https://github.com/pmeier/pystiche@v0.5.1 for URL https://github.com/pmeier/pystiche@v0.5.1

I instead used pip install git+https://github.com/pmeier/pystiche@v0.5.1

This is my pip version to help debugging the issue:

pip --version
pip 20.2.2 from /home/bluebox/miniconda3/envs/pystiche_review/lib/python3.6/site-packages/pip (python 3.6)
missing documentation

pystiche.meta, pystiche.misc.build_complex_obj_repr, pystiche.ops.functional.value_range_loss, some functions in pystiche.image, most functions in pystiche.image.transforms didn't have any documentation beyond input/return type information

issue with running the example

I had issue running the first example example_nst_with_pystiche.py downloaded from https://pystiche.readthedocs.io/en/stable/_downloads/36c76ed9ccf74ecc12b41424d1ea227a/example_nst_with_pystiche.py which was linked from the end of this page: https://pystiche.readthedocs.io/en/stable/galleries/examples/beginner/example_nst_with_pystiche.html#sphx-glr-galleries-examples-beginner-example-nst-with-pystiche-py

The exact error was:

Traceback (most recent call last):
  File "example_nst_with_pystiche.py", line 116, in <module>
    images.download()
  File "/home/bluebox/miniconda3/envs/pystiche_review/lib/python3.6/site-packages/pystiche/data/collections/download.py", line 200, in download
    image.download(root=root, overwrite=overwrite)
  File "/home/bluebox/miniconda3/envs/pystiche_review/lib/python3.6/site-packages/pystiche/data/collections/download.py", line 144, in download
    raise FileExistsError(msg)
FileExistsError: bird1.jpg with a different MD5 hash already exists in /home/bluebox/.cache/pystiche. If you want to overwrite it, set overwrite=True.

I have never installed pystiche on this machine before. To double-confirm, I removed rm -rf /home/bluebox/.cache/pystiche and reran the script again, but the error is the same.

To help debug:

python --version
Python 3.6.12 :: Anaconda, Inc.

pip list|grep torch
torch             1.6.0
torchvision       0.7.0

pip list|grep pystiche
pystiche          0.5.1
issue with running unit tests

Running tests didn't work out of the box. I was following the instructions in CONTRIBUTING.rst: https://github.com/pmeier/pystiche/blob/v0.5.1/CONTRIBUTING.rst#tests

See the log for details on the environment, commands used, exact output, etc: https://gist.github.com/soumith/70f989dbd9d9d5702cbb24143e88755c

I decided to try get these to work manually by calling pytest . and fixing issues. I had to do the following:

pip install dill
pip install https://github.com/pmeier/pyimagetest/archive/v0.2.zip # i.e. the previous version of pyimagetest
pip install pillow-affine
pip install pytest-subtests

After this, pytest . started working, but one test failed: https://gist.github.com/soumith/690f8fdc9a224f85a0484a04f65d40ec pytest . --skip-slow worked to completion

pmeier

comment created time in 5 days

pull request commentpytorch/builder

Downgrade MKL to 2020.0

This is only for manywheel right?

wrt conda, we currently have mkl >= 2018. We might not be able to change it to add the < 2020.1 because that would downgrade other conda packages down, such as numpy/scipy. realized this, so wanted to flag before I forgot.

malfet

comment created time in 5 days

push eventpytorch/hub

Soumith Chintala

commit sha 5eac8f0358f5ce9c4962f816ffa87b3ec22ed6aa

Update snakers4_silero-models_stt.md

view details

push time in 5 days

issue commentpytorch/pytorch

'package not found error',while installing pytorch on anaconda python 3.7

@brando90 looks like you are on a ppc64le which we don't have binaries for on the default pytorch channel (maybe IBM has them in their ppc64le channel)

Luiz-HSSD

comment created time in 6 days

PR closed pytorch/pytorch

Reviewers
Add Deep Learning with Pytorch book(free copy) to Resource open source

Fixes #{issue number}

+1 -0

3 comments

1 changed file

DFRICHARD

pr closed time in 6 days

pull request commentpytorch/pytorch

Add Deep Learning with Pytorch book(free copy) to Resource

let's rely on pytorch.org to disseminate this information, as it might get outdated.

DFRICHARD

comment created time in 6 days

pull request commentpytorch/hub

Add Silero Speech-To-Text models

apparently we've moved to a once-per-day update of the site now. it should be live by tomorrow morning.

snakers4

comment created time in 6 days

pull request commentpytorch/hub

Add Silero Speech-To-Text models

thanks for the great contribution. it should go live sometime today, maybe in a couple of hours.

snakers4

comment created time in 6 days

push eventpytorch/hub

Alexander Veysov

commit sha 9146b39dec42f6ff7080ef0f756b956b5dd67dcd

Add Silero Speech-To-Text models (#153) * Add Silero Speech-To-Text models * Move the MD file to root folder * Fix util imports, fix models.yml path issues * Add TorchAudio to the test environment * Add -y to TorchAudio install * Add OmegaConf installation to build * Add dependencies installation, add one file example

view details

push time in 6 days

PR merged pytorch/hub

Add Silero Speech-To-Text models

Please kindly review our Speech-To-Text models

+87 -0

6 comments

5 changed files

snakers4

pr closed time in 6 days

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentpytorch/hub

Add Silero Speech-To-Text models

+---+layout: hub_detail+background-class: hub-background+body-class: hub+category: researchers+title: Silero Speech-To-Text Models+summary: A set of compact enterprise-grade pre-trained STT Models for multiple languages.+image: silero_logo.jpg+author: Silero AI Team+tags: [audio, scriptable]+github-link: https://github.com/snakers4/silero-models+github-id: snakers4/silero-models+featured_image_1: silero_stt_model.jpg+featured_image_2: silero_imagenet_moment.png+accelerator: cuda-optional+---++```python+import torch+import zipfile+import torchaudio+from glob import glob+# see https://github.com/snakers4/silero-models for utils and more examples++device = torch.device('cpu')  # gpu also works, but our models are fast enough for CPU+model, decoder, utils = torch.hub.load(github='snakers4/silero-models',

would this require omegaconf and torchaudio to be installed?

if so, you should add a cell like in https://github.com/pytorch/hub/blob/master/nvidia_deeplearningexamples_waveglow.md#example which pip installs these extra packages.

that will make the thing instantly run in Google Colab, and is really valuable!

snakers4

comment created time in 7 days

Pull request review commentpytorch/hub

Add Silero Speech-To-Text models

+---+layout: hub_detail+background-class: hub-background+body-class: hub+category: researchers+title: Silero Speech-To-Text Models+summary: A set of compact enterprise-grade pre-trained STT Models for multiple languages.+image: silero_logo.jpg+author: Silero AI Team+tags: [audio, scriptable]+github-link: https://github.com/snakers4/silero-models+github-id: snakers4/silero-models+featured_image_1: silero_stt_model.jpg+featured_image_2: silero_imagenet_moment.png+accelerator: cuda-optional+---++```python+import torch+import zipfile+import torchaudio+from glob import glob+# see https://github.com/snakers4/silero-models for utils and more examples++device = torch.device('cpu')  # gpu also works, but our models are fast enough for CPU+model, decoder, utils = torch.hub.load(github='snakers4/silero-models',+                                       model='silero_stt',+                                       device=device,+                                       force_reload=True)+(read_batch,+ split_into_batches,+ read_audio,+ prepare_model_input) = utils  # see function signature for details++torch.hub.download_url_to_file('http://www.openslr.org/resources/83/midlands_english_female.zip',

instead of a zip file, extracting it and then running a batch of wav files through, it would be much nicer and more illustrative of downloading a single wav file and processing it through.

like:

torch.hub.download_url_to_file('some download url for speech.wav', dst='speech.wav)

input = prepare_model_input(glob('midlands_english_female/*.wav'), device=device)

output = model(input)
snakers4

comment created time in 7 days

PullRequestReviewEvent
PullRequestReviewEvent

issue commentpyOpenSci/software-review

pystiche: A Framework for Neural Style Transfer

hey everyone, still working on the review, but I should have something by EOD on the 23rd.

pmeier

comment created time in 7 days

pull request commentfacebookresearch/pyrobot

bug fix

Currently pix_to_3dpt & get_current_pcd function in habitat wrapper returns point in global frame while in locobot, it returns it in robot base frame

Ok, yea then when we were implementing it for Habitat, we completely had the wrong idea.

How does one then transform the point cloud into the global frame after it is returned by get_current_pcd?

Dhiraj100892

comment created time in 7 days

Pull request review commentfacebookresearch/pyrobot

bug fix

 def pix_to_3dpt(self, rs, cs, in_cam=False, initial_rotation=None):         pts = self._cam2pyrobot(pts, initial_rotation=initial_rotation)         return pts, colors -     def _cam2pyrobot(self, pts, initial_rotation=None):         """         here, points are  given in camera frame         the thing to do next is to transform the points from camera frame into the         global frame of pyrobot environment         This does not translate to what habitat thinks as origin,         because pyrobot's habitat-reference origin is `self.agent.init_state`-        So, CAMERA frame -> HABITAT frame -> PYROBOT frame+        So, CAMERA frame -> HABITAT frame -> PYROBOT frame (robot base frame)

is this correct?

@kalyanvasudev said (when we were debugging last time) that PYROBOT frame is basically the global coordinate frame in the pyrobot environment.

The line above this also describes it differently from robot base frame:

the thing to do next is to transform the points from camera frame into the
        global frame of pyrobot environment
Dhiraj100892

comment created time in 7 days

PullRequestReviewEvent

issue commentkakaobrain/torchgpipe

Dual-license as BSD3 for PyTorch integration

thanks so much @sublee , great news.

pritamdamania87

comment created time in 10 days

issue commentpytorch/builder

Where are the windows CUDA builds?

@peterjc123 would know.

scopatz

comment created time in 11 days

push eventfacebookresearch/pyrobot

Soumith Chintala

commit sha 0d207bcaf1b79a8eb62c0e9bbdc4b2664cdcf4e4

add initial_rotation passthrough to Habitat camera backend of get_current_pcd

view details

push time in 13 days

issue commentd2l-ai/d2l-en

Maybe our DCGAN have extra nn.BatchNorm2d(ndf)?

Unnecessary means that it is OK to add the layer if we use D_block way to simplify code?

Yes you can add if you want.

And BTW, in my stupid code, I found pytorch is slower than mxnet. Can you help me improve the pytorch code...

Maybe PyTorch is slower than MXNet :)

StevenJokes

comment created time in 19 days

startedfacebookresearch/fairscale

started time in 19 days

issue commentkakaobrain/torchgpipe

Dual-license as BSD3 for PyTorch integration

hey, wanted to second the excitement.

It's also important to recognize and acknowledge contributions. So, from the PyTorch side, we will commit to adding in the release notes (cc: @pritamdamania87 @mrshenli ) that the code was forked from kakaobrain/torchgpipe and recognize your efforts. If you would like additional ways of acknowledgement, happy to help with that as well.

pritamdamania87

comment created time in 19 days

pull request commentpytorch/tutorials

DCGAN's Discriminator : add `nn.BatchNorm2d(ndf)` to fix #1126

hey, BatchNorm seems unnecessary for the first layer, as the inputs to the first Conv2d are already zero-mean and unit std, and the Conv2d is initialized (initially in a random draw) in a way that the output of the Conv2d roughly maintains zero-mean and unit std.

This is why we avoided putting batchnorm in the first layer in our original code / paper

StevenJokes

comment created time in 20 days

issue commentd2l-ai/d2l-en

Maybe our DCGAN have extra nn.BatchNorm2d(ndf)?

hey, BatchNorm seems unnecessary for the first layer, as the inputs to the first Conv2d are already zero-mean and unit std, and the Conv2d is initialized (initially in a random draw) in a way that the output of the Conv2d roughly maintains zero-mean and unit std.

This is why we avoided putting batchnorm in the first layer in our original code / paper

StevenJokes

comment created time in 20 days

issue closedpytorch/builder

Where are Python 3.8 manywheel binaries built for nightlies?

Hi!

Thanks for making this repo public, it's very helpful. I've noticed that PyTorch is pushing pip manywheel nightlies for python 3.8, but it seems like 3.8 is missing from the configuration in cron/ (e.g., 3.8 isn't defined here https://github.com/pytorch/builder/blob/master/cron/build_multiple.sh#L55 , among other places)

Am I missing something, or is the cron configuration to build 3.8 pip manywheels just not pushed yet?

Thanks in advance!

closed time in 20 days

nelson-liu

issue commentpytorch/builder

Where are Python 3.8 manywheel binaries built for nightlies?

the cron folder is a bit outdated. now the binaries are triggered by CircleCI in this folder: https://github.com/pytorch/pytorch/tree/master/.circleci

Here's 3.8 spec: https://github.com/pytorch/pytorch/blob/master/.circleci/cimodel/data/binary_build_data.py#L36 and https://github.com/pytorch/pytorch/blob/master/.circleci/cimodel/data/dimensions.py#L14

Sorry, that folder is a bit over-engineered

nelson-liu

comment created time in 20 days

pull request commentpytorch/hub

Fix missing unidecode package

thanks!

apthagowda97

comment created time in 25 days

push eventpytorch/hub

Aptha Gowda

commit sha 28577211ed58c9d6678c9e136040db52836b926b

installed unidecode before loading tacotron2 (#151)

view details

push time in 25 days

PR merged pytorch/hub

Fix missing unidecode package

Unidecode package was required for loading tacotron2 but it was installed later.

+8 -7

1 comment

1 changed file

apthagowda97

pr closed time in 25 days

PullRequestReviewEvent

push eventpytorch/examples

Natalia Gimelshein

commit sha 49e1a8847c8c4d8d3c576479cb2fe2fd2ac583de

use reshape instead of view in accuracy

view details

push time in a month

PR merged pytorch/examples

use reshape instead of view in accuracy

Per title. Layout of pred is now correctly propagated so that correct is not contiguous.

+1 -1

0 comment

1 changed file

ngimel

pr closed time in a month

PullRequestReviewEvent
PullRequestReviewEvent

issue commentpyOpenSci/software-review

pystiche: A Framework for Neural Style Transfer

i'll send in my review by that deadline. thanks for the heads-up!

pmeier

comment created time in a month

issue commentpytorch/pytorch

Using external memory allocator with PyTorch

@datametrician i'd let @csarofeen @ptrblck and @ngimel figure out next steps from the PyTorch side. I believe they are figuring out the subtle details and exact plan

jakirkham

comment created time in a month

push eventpytorch/tutorials

Sayantan Das

commit sha 6ab3a3771b203395bda42d244fdff5f9bd38bb09

Corrected model.resnet50() spelling (#1139) Spelling mistake led to errors for beginners.

view details

push time in a month

PR merged pytorch/tutorials

Small PR : Corrected model.resnet50() spelling

Spelling mistake led to errors for beginners.

Reproduce

  • Browse to https://pytorch.org/tutorials/intermediate/model_parallel_tutorial.html#apply-model-parallel-to-existing-modules . second line.
+2 -2

3 comments

1 changed file

ucalyptus

pr closed time in a month

pull request commentpytorch/builder

common: Comment out pruning for CUDA 11

cc: @zasdfgbnm @csarofeen

if we can't prune, we can't ship CUDA 11 binaries to PyPI, and I know CuBLAS/CuDNN releases are really long, so worth getting a head-start on this

seemethere

comment created time in a month

pull request commentpytorch/builder

Update MKL version to 2020.1

@malfet we should consider reverting this, in that case.

Basically, MKL 2019 / 2020 / 2020.1 don't really give that much extra perf benefit. The fact that Intel removed this workaround makes us need to evaluate our strategy here, honestly. Should we consider switching to OpenBLAS (and write our own FFT functions)?

cc: @ngimel @jamesr66a @gottbrath

malfet

comment created time in a month

PullRequestReviewEvent
CommitCommentEvent

pull request commentpytorch/builder

manywheel: Add arch 8.0 to CUDA 10.0 builds

you mean to CUDA 11.0 builds

seemethere

comment created time in a month

push eventpytorch/pytorch.github.io

Joseph Spisak

commit sha ea2b1e60a66c4b02c00e2796689d55df3e090373

Create ray (#460)

view details

push time in a month

PR merged pytorch/pytorch.github.io

Reviewers
Create ray

Adding Ray to the ecosystem

cc @CamiWilliams

+10 -0

1 comment

1 changed file

jspisak

pr closed time in a month

issue commentpytorch/pytorch

Using external memory allocator with PyTorch

@jrhemstad the catch22 is not when a user picks a bad custom allocator. It is when a library that the user is using opaquely picks a bad custom allocator. In that situation, the user might not even be aware that this is happening under the hood, they only see bad performance and attribute it ot "something something pytorch bad..."

jakirkham

comment created time in a month

Pull request review commentpytorch/hub

Update pytorch_fairseq_translation.md

 author: Facebook AI (fairseq Team) tags: [nlp] github-link: https://github.com/pytorch/fairseq/ github-id: pytorch/fairseq+featured_image_1: no-image+eatured_image_2: no-image

this has to be featured_image_2

andresruizfacebook

comment created time in a month

PullRequestReviewEvent

issue commentpytorch/pytorch

Using external memory allocator with PyTorch

I'm generally supportive of the direction, but we do need to figure out some details.

What @colesbury said would be technical needs. How it might manifest is in the form of import side-effects.

import foo    # here, foo imports and sets torch to an external allocator that is suboptimal
import torch

out = mymodel(x) # this line runs much slower if `foo` is imported because of the suboptimal overriden allocator

Detecting a suboptimal allocator I think isn't technically possible (other than profiling). So, when an external allocator is enabled, a reasonable thing for pytorch to do is to enable warnings to the user that something suboptimal might happen.

My concern is around whether we have an external memory allocator that we all agree is reasonably performant, that we allow overrides to (or allow overrides to without emitting a warning).

Any thoughts?

jakirkham

comment created time in a month

pull request commentpytorch/pytorch

[NVFuser] Enable E2E BCast-PWise-Reduction fusions

no, it was landing, it's done now.

csarofeen

comment created time in a month

startedkarpathy/minGPT

started time in a month

pull request commentpytorch/builder

llvm: Add llvm docker build

why are we using llvm?

seemethere

comment created time in a month

issue commentpytorch/pytorch

Cuda error: no kernel image is available for execution on the device

@trevorgribble CUDA Capability 3.0 is not actually supported by PyTorch, as we use __shfl instructions at places, which only came out starting CUDA capability 3.5.

legoritma

comment created time in 2 months

push eventpytorch/examples

mattip

commit sha a715dd8d7168462c5c622369baef3e19b7feadd9

Add a daily CI run and an action to open an issue

view details

mattip

commit sha 4b119d735b802453479d739bf823f3f7d8d5d422

add warning to README

view details

push time in 2 months

pull request commentpytorch/examples

Run CI daily

thanks @mattip !

mattip

comment created time in 2 months

PR merged pytorch/examples

Run CI daily

xref pytorch/pytorch#32004

Run github actions CI once a day (at 3:00) and if it fails, open an issue. mattip/pytorch-examples#21 is a forced example of such an issue. A few things to note:

  • The main purpose of this action is to consistently test this repo against pytorch nightly builds, to see if some new pytorch "feature" breaks one of the examples.
  • Anyone who forks this repo will, by default, get this scheduled action. I added a note to the README. We could also add a step to emit an issue if {{ github.repository_owner != 'pytorch' }} which would hopefully get people's attention :). By default forks do not have issues, so I am not sure what would happen. Forking this repo does seem like a popular thing to do: there are 64.k forks. Searching for a way to disable via the yml led me to this discussion which seems to indicate it cannot be done right now. Another option would be to move from github actions (which are available on all public repos) to travis or circleci, which would require action on part of the fork to enable it. I can pivot this PR to do that, but a admin would have to enable another CI service.
  • I hope someone is getting mails on all new issues in this repo, otherwise the issue will not get much attention.
+20 -5

7 comments

2 changed files

mattip

pr closed time in 2 months

pull request commentpytorch/pytorch

Add an absolute pixel coordinates option to grid_sample

@gchanan @houseroad what does one have to do to get the Backward Compatibility tests to pass? this PR adds a new option at the end with a default, so it technically should be backward compatible.

bnehoran

comment created time in 2 months

Pull request review commentpytorch/pytorch

Add an absolute pixel coordinates option to grid_sample

 struct ComputeLocation<scalar_t, GridSamplerPadding::Border, align_corners>   using ComputeLocationBase<scalar_t, align_corners>::ComputeLocationBase;    inline Vec apply(const Vec &in) const {-    return clip_coordinates(unnormalize(in));+    Vec res;+    if (pixel_coords) {+      res = in;+    } else {+      res = unnormalize(in);+    }+    return clip_coordinates(res);   }    inline std::pair<Vec, Vec> apply_get_grad(const Vec &in) const {-    Vec res, grad_clip;-    std::tie(res, grad_clip) = clip_coordinates_get_grad(unnormalize(in));-    return std::make_pair(res, grad_clip & Vec(scaling_factor));+    Vec res, grad_clip, grad;+    if (pixel_coords) {+      res = in;+      grad = Vec(1);

just checking that this is not Vec(scaling_factor)

bnehoran

comment created time in 2 months

Pull request review commentpytorch/builder

manywheel: Move to manylinux2014

 # syntax = docker/dockerfile:experimental ARG BASE_CUDA_VERSION=10.1-FROM centos:7 as base+FROM quay.io/pypa/manylinux2014_x86_64 as python+# remove unncessary python versions+RUN rm -rf /opt/python/cp27* /opt/_internal/cpython-2.7*+RUN rm -rf /opt/python/cp35* /opt/_internal/cpython-3.5*++FROM python as libcrypt+RUN mkdir -p /libcrypt && find /usr/local/lib -type f -name "libcrypt*" -exec cp -L {} /libcrypt \;+RUN mkdir -p /libcrypt64 && find /usr/lib64 -type f -name "libcrypt*" -exec cp -L {} /libcrypt64 \;+# Allows us to restore all the links, since copying them wasn't tenable+RUN for link in $(find / -name "libcrypt*" -type l); do echo "ln -sf $(readlink -f ${link})  ${link}" >> /relink_libcrypt; done +FROM nvidia/cuda:${BASE_CUDA_VERSION}-devel-centos7 as base+ARG BASE_CUDA_VERSION=10.1 ENV LC_ALL en_US.UTF-8 ENV LANG en_US.UTF-8 ENV LANGUAGE en_US.UTF-8--RUN yum install -y wget curl perl util-linux xz bzip2 git patch which perl+RUN yum install -y \+        automake \+        bison \+        bzip2 \+        curl \+        diffutils \+        file \+        git \+        openssl \+        openssl-devel \+        patch \+        perl \+        unzip \+        util-linux \+        wget \+        which \+        xz \+        yasm RUN yum install -y yum-utils centos-release-scl RUN yum-config-manager --enable rhel-server-rhscl-7-rpms RUN yum install -y devtoolset-7-gcc devtoolset-7-gcc-c++ devtoolset-7-gcc-gfortran devtoolset-7-binutils

dont care as long as it compiles. afaicr devtoolset9 (gcc-9) didn't compile pytorch when we checked a long time ago. it probably does, now.

seemethere

comment created time in 2 months

pull request commentpytorch/builder

Add cubinsizes.py tool

this is pretty cool!

malfet

comment created time in 2 months

PR closed pytorch/examples

Add fizz/buzz example.

Add fizz/buzz example, you can use --num-digits to assign the training/testing digit.

+161 -0

1 comment

3 changed files

i25ffz

pr closed time in 2 months

pull request commentpytorch/examples

Add fizz/buzz example.

thanks for the contribution, but it doesn't seem like a good representative example of anything people would want to do, and hence learn from

i25ffz

comment created time in 2 months

push eventpytorch/examples

vfdev

commit sha 8ec17a42412c3cb76bd8e2392e8a4fea5718ebb0

Updated CI badge

view details

push time in 2 months

pull request commentpytorch/examples

Updated CI badge

thanks!

vfdev-5

comment created time in 2 months

PR merged pytorch/examples

Updated CI badge

Minor fix of github CI badge for better rendering.

+1 -1

0 comment

1 changed file

vfdev-5

pr closed time in 2 months

pull request commentpytorch/builder

Remove special case for CUDA-10.2

conda binaries shouldn't have been broken, only the wheels were broken.

malfet

comment created time in 2 months

push eventpytorch/pytorch.github.io

Nikita Shulga

commit sha 2153a097cc61b65b1fc517b6c2a7dbf6f6318860

Add 1.4.0 and 1.5.0 to previous versions (#439)

view details

push time in 2 months

startedpytorch/elastic

started time in 2 months

issue commentpytorch/pytorch

PyTorch should hide CUDNN symbols in libtorch.so

-Wl,exclude-libs is a reasonable and easy fix, we should do it.

Presumably, has to be inserted here: https://github.com/pytorch/pytorch/blob/6e4f99b0633f704f8c7bf3bb3b9733680f5dc18b/cmake/public/cuda.cmake#L256-L271

follow along USE_STATIC_CUDNN=1

nvpohanh

comment created time in 2 months

issue commentpytorch/pytorch

[GitHub] Add pull request template

Adding a section to CONTRIBUTING sounds better, or a PR Template that is extremely simple, for example like literally having one line that says "Fixes ### " so that people don't forget to list the issues it fixes.

SplitInfinity

comment created time in 2 months

issue commentpytorch/builder

PyPI Packaging for C++ Extension

this is what we do, instead of using auditwheel: https://github.com/pytorch/builder/blob/master/manywheel/build_common.sh#L231-L323

  • we unzip the wheel
  • we copy over the dependencies
  • we patch the RPATH for these dependencies
  • we generated a new RECORD file in the wheel with the new files + hashes
  • we zip back the wheel
Queuecumber

comment created time in 2 months

pull request commentpytorch/pytorch

Add an absolute pixel coordinates option to grid_sample

i'll defer the review to @fmassa

bnehoran

comment created time in 2 months

Pull request review commentpytorch/pytorch

Preserve CUDA gencode flags

 def _check_capability():             elif CUDA_VERSION <= 9000 and major >= 7 and minor >= 5:                 warnings.warn(incorrect_binary_warn % (d, name, 10000, CUDA_VERSION)) +def _check_cubins():+    if torch.version.cuda is None:  # on ROCm we don't want this check+        return+    arch_list = get_arch_list()+    if len(arch_list) == 0:+        return+    supported_sm = [int(arch.split('_')[1]) for arch in arch_list if 'sm_' in arch]+    for idx in range(device_count()):+        cap_major, cap_minor = get_device_capability(idx)+        capability = cap_major * 10 + cap_minor+        # NVIDIA GPU compute architectures are backward compatible within 5 minor revisions versions+        supported = any([capability >= sm and capability - (sm // 5) * 5 < 5 for sm in supported_sm])+        if not supported:+            warnings.warn(f"{get_device_name(idx)} with capability {capability} is not compatible with {arch_list}")

since you are already landing this PR, can you please improve the warning here. Maybe to the effect of:

"With the current PyTorch installation, [device name] with CUDA capability [capability] is not compatible. The current PyTorch install supports CUDA capabilities [arch_list]. If you want to use the [device name] GPU with PyTorch, you might have to compile PyTorch from source"

OR

"With the current PyTorch installation, [device name] with CUDA capability [capability] is not compatible. The current PyTorch install supports CUDA capabilities [arch_list]. If you want to use the [device name] GPU with PyTorch, you might have to download and install the all-GPU PyTorch wheels from the following page [URL]"

malfet

comment created time in 3 months

Pull request review commentpytorch/builder

manywheel: Simplify cuda arch, remove 3.7 and 6.1

 else     echo "CUDA $CUDA_VERSION Detected" fi -export TORCH_CUDA_ARCH_LIST="3.7+PTX;5.0"-if [[ $CUDA_VERSION == "9.0" ]]; then-    export TORCH_CUDA_ARCH_LIST="$TORCH_CUDA_ARCH_LIST;6.0;7.0"-elif [[ $CUDA_VERSION == "9.2" ]]; then-    export TORCH_CUDA_ARCH_LIST="$TORCH_CUDA_ARCH_LIST;6.0;6.1;7.0"-    # ATen tests can't build with CUDA 9.2 and the old compiler used here-    EXTRA_CAFFE2_CMAKE_FLAGS+=("-DATEN_NO_TEST=ON")-elif [[ $CUDA_VERSION == "10.0" ]]; then-    export TORCH_CUDA_ARCH_LIST="$TORCH_CUDA_ARCH_LIST;6.0;6.1;7.0;7.5"-    # ATen tests can't build with CUDA 10.0 maybe???? (todo) and the old compiler used here-    EXTRA_CAFFE2_CMAKE_FLAGS+=("-DATEN_NO_TEST=ON")-elif [[ $CUDA_VERSION == "10.1" ]]; then-    export TORCH_CUDA_ARCH_LIST="$TORCH_CUDA_ARCH_LIST;6.0;6.1;7.0;7.5"-    # ATen tests can't build with CUDA 10.1 maybe???? (todo) and the old compiler used here-    EXTRA_CAFFE2_CMAKE_FLAGS+=("-DATEN_NO_TEST=ON")-elif [[ $CUDA_VERSION == "10.2" ]]; then-    export TORCH_CUDA_ARCH_LIST="$TORCH_CUDA_ARCH_LIST;6.0;6.1;7.0;7.5"-    # ATen tests can't build with CUDA 10.1 maybe???? (todo) and the old compiler used here-    EXTRA_CAFFE2_CMAKE_FLAGS+=("-DATEN_NO_TEST=ON")-else-    echo "unknown cuda version $CUDA_VERSION"-    exit 1-fi+export TORCH_CUDA_ARCH_LIST="5.0"

I'd rather keep 3.7+PTX than 5.0. 3.7 is used much more than 5.2 / 5.x

seemethere

comment created time in 3 months

push eventpytorch/examples

mattip

commit sha 3c032e84c9e9e1c3e2500a5a8c3b5b6bd6ca3a8c

Add CI to run examples via github action

view details

mattip

commit sha 6c8e2bab4d45f2386929c83bb4480c18d2b660fd

fix warnings and failures

view details

mattip

commit sha e7870c1fd4706174f52a796521382c9342d4373f

remove tput, causes errors in CI

view details

push time in 3 months

PR merged pytorch/examples

WIP: Add CI to run examples via github action

Add github action to run examples on CPU

+43 -8

0 comment

6 changed files

mattip

pr closed time in 3 months

issue closedpytorch/pytorch

reinforcement learning dataloading and algorithms

🚀 Feature

Implement a dataloading functionality for reinforcement learning state, action pairs, with assigned policy scores, transitional probabilities and rewards. Implement a set of gradient algorithms (qlearning, actor-critic, etc) in order to populate policy maps and use them or export them for other models.

Motivation

I feel there is a gap in pytorch, which doesn't address a relatively interesting field of machine learning. I understand that reinforcement learning falls outside deep learning, and that simple examples of DQN already exist in tutorials, blogs and such, but I feel that impementing a GPU and CPU based dataloader which is easy to use and understand, combined with the algorithms that calculate the policy values and maps, would benefit pytorch as well as those who already use pytorch for a variety of reasons. I think that there's a lot of software domains where this would be applicable.

Pitch

  1. develop a dataloader class or set of classes which deal with deterministic state-action spaces and with probabilistic state-action spaces.
  2. develop a set of reinforcement learning algorithm classes which can be applied to those dataloaders, and populate policy values, based on rewards and such

Alternatives

No much really.

Additional context

I think that pytorch has grown to be a wonderful tool used in so many different places, and that reinforcement learning seems to be missing. I know theres a tutorial for DQN, and I know that a lot of the algorithms are trivial and simple, yet it seems to me that users rely on pytorch as a simple and easy to use tool for machine learning, and as such don't always have the time (or know-how) on how to implement that functionality.

Just to be clear I propose I implement this with some guidance, as I've already done this in C, C++ and Python before. I've got a few ideas on how to do this with the underlying C++ aten and autograd libraries and then move on to the python bindings.

cc @SsnL

closed time in 3 months

alexge233

push eventpytorch/examples

Matti Picus

commit sha 49ec0bd72b85be55579ae8ceb278c66145f593e1

tweak running examples without cuda (#794) * tweak running examples without cuda * rework dry_run handling in mnist, mnist_hogwild

view details

push time in 3 months

PR merged pytorch/examples

tweak running examples without cuda

Tweak the run_python_examples.sh script to complete in ~5 minutes (on my machine) without CUDA. Once this is merged, we can consider adding #704 to actually run the examples as part of CI.

TBD: how do we get a pre-release version of torch/torchvision in order to test the latest HEAD? Perhaps it should be a pytorch/pytorch cron job that runs once a day or once a week?

ping @jlin27

+120 -57

3 comments

10 changed files

mattip

pr closed time in 3 months

push eventsoumith/tacotron2

Soumith

commit sha 79b9227ff687fd79a3de450c91aab27c53e264b3

install dependencies

view details

Soumith

commit sha ccde042c98d141dc2f9e8b42eedbf6a754474203

shorten dataset

view details

push time in 3 months

fork soumith/tacotron2

Tacotron 2 - PyTorch implementation with faster-than-realtime inference

fork in 3 months

pull request commentpytorch/pytorch

[WIP] `torch.nn.Module` Shape Inference proposal

I've taken a look at the PR, sorry for the delay.

It looks to be in a reasonably good shape, and covers my original outline of requirements pretty well.

At first, I also thought about creating a LazyModule and LazyLinear classes, but since that conflicted with @soumith proposal I went to directly edit the modules themselves. However, now that we have to throw #29523 into the mix maybe it is the best option.

Generally, my point is about explicitness -- the user has to be very aware that they are doing this shape inference scheme, which LazyLinear does cover. In fact LazyLinear() might be easier to write, than nn.Linear(nn.INFER, ...).

emcastillo

comment created time in 3 months

more