I want to create a new account of Pytorch forums, but the web always says: " something goes wrong, please try forget passwords link..."

can you see if it works now. we fixed it. please reopen if it doesn't work

cc: @ptrblck

@seemethere can you check on this, maybe we've regressed?

A, B, C, D are commits

• master branch -> A -> B -> C
• ghstack invoked, and PRs created for A, B, C
• commit D created, and rebased so that master branch -> D -> A -> B -> C
• ghstack invoked

in this case, the base branch of A is not moved to be D.

Here's the rage log:

https://gist.github.com/soumith/95c36efdb06cd23d98a9727d20042c51

version ghstack v0.2.1

Tried updating to ghstack v0.3.0 and running ghstack again, but it didn't do anything

false report. ignore.

the errors should be the default, but let's say F.one_hot(..., relaxed=True) or something can do the more relaxed behavior

A, B, C, D are commits

• master branch -> A -> B -> C
• ghstack invoked, and PRs created for A, B, C
• commit D created, and rebased so that master branch -> D -> A -> B -> C
• ghstack invoked

in this case, the base branch of A is not moved to be D.

Here's the rage log:

https://gist.github.com/soumith/95c36efdb06cd23d98a9727d20042c51

version ghstack v0.2.1

@suo sorry i didn't see your github comments. I ended up clicking the land button a while ago, and it landed it in the night.

got a critical lint issue, possibly from already committed code

This file should only include printable US-ASCII bytes.
Use Unicode escapes like `\u` to represent disallowed values.

torch/csrc/jit/codegen/cuda/lower_unroll.h

Package Review

Please check off boxes as applicable, and elaborate in comments below. Your review is not limited to these topics, as described in the reviewer guide

• [x] As the reviewer I confirm that there are no conflicts of interest for me to review this work (If you are unsure whether you are in conflict, please speak to your editor before starting your review).

Documentation

The package includes all the following forms of documentation:

• [x] A statement of need clearly stating problems the software is designed to solve and its target audience in README
• [x] Installation instructions: for the development version of package and any non-standard dependencies in README
• [x] Vignette(s) demonstrating major functionality that runs successfully locally
• [?] Function Documentation: for all user-facing functions
  • not all but ~90% -- see review comments below
• [ ] Examples for all user-facing functions
  • I didn't see inline doc examples for almost any function. Is this the expectation of this review line item?
• [x] Community guidelines including contribution guidelines in the README or CONTRIBUTING.
• [x] Metadata including author(s), author e-mail(s), a url, and any other relevant metadata e.g., in a setup.py file or elsewhere.

Readme requirements The package meets the readme requirements below:

• [x] Package has a README.md file in the root directory.

The README should include, from top to bottom:

• [x] The package name
• [x] Badges for continuous integration and test coverage, a repostatus.org badge, and any other badges. If the README has many more badges, you might want to consider using a table for badges: see this example. Such a table should be more wide than high. (Note that the badge for pyOpenSci peer-review will be provided upon acceptance.)
• [x] Short description of goals of package, with descriptive links to all vignettes (rendered, i.e. readable, cf the documentation website section) unless the package is small and there’s only one vignette repeating the README.
• [x] Installation instructions
• [x] Any additional setup required (authentication tokens, etc)
• [ ] Brief demonstration usage
• [x] Direction to more detailed documentation (e.g. your documentation files or website).
• [x] If applicable, how the package compares to other similar packages and/or how it relates to other packages
• [ ] Citation information

Usability

Reviewers are encouraged to submit suggestions (or pull requests) that will improve the usability of the package as a whole. Package structure should follow general community best-practices. In general please consider:

• [x] The documentation is easy to find and understand
• [x] The need for the package is clear
• [?] All functions have documentation and associated examples for use
  • ~90% functions have documentation, and the repository examples cover a lot of beginner to complex cases, so I consider this still sufficient

Functionality

• [x] Installation: Installation succeeds as documented.
• [ ] Functionality: Any functional claims of the software been confirmed.
  • see my notes below, I had trouble running the example out of the box
• [x] Performance: Any performance claims of the software been confirmed.
  • no performance claims were made by software
• [?] Automated tests: Tests cover essential functions of the package and a reasonable range of inputs and conditions. All tests pass on the local machine.
  • see notes below, had trouble running tests out of the box using instructions. I had to manually install the right packages to get tests working.
• [x] Continuous Integration: Has continuous integration, such as Travis CI, AppVeyor, CircleCI, and/or others.
• [?] Packaging guidelines: The package conforms to the pyOpenSci packaging guidelines.
  • the link to the packaging guidelines is broken

For packages co-submitting to JOSS

• [x] The package has an obvious research application according to JOSS's definition in their submission requirements.

Note: Be sure to check this carefully, as JOSS's submission requirements and scope differ from pyOpenSci's in terms of what types of packages are accepted.

The package contains a paper.md matching JOSS's requirements with:

• [x] A short summary describing the high-level functionality of the software
• [x] Authors: A list of authors with their affiliations
• [x] A statement of need clearly stating problems the software is designed to solve and its target audience.
• [ ] References: with DOIs for all those that have one (e.g. papers, datasets, software).
  • the references section is empty, that doesn't look right: https://github.com/pmeier/pystiche/blob/v0.5.1/paper.md

Final approval (post-review)

• [ ] The author has responded to my review and made changes to my satisfaction. I recommend approving this package.

Estimated hours spent reviewing: 1.5

Review Comments

The package makes the whole suite of neural style transfer work much easier to comprehend and reliably run. The documentation was nice, the examples were a breeze to read through. There's a lot of value in such a package and it's obvious that a lot of effort and tasteful design have been put into it's development.

Bigger points:

• The overall API documentation as well as examples are in really good shape. I don't have any real comments here. In-line examples for some of the functional operators (in pystiche.ops and pystiche.ops.functional) would have been nice.
• I reviewed v0.5.1 (latest stable release) and there seem to be teething problems with the environment I setup (a fresh conda env with python 3.6, and then following package instructions). I've documented it in detail enough that the author should be able to resolve this quickly.
  • Unfortunately, I wasn't able to run the examples locally due to an image hash issue that I've documented below. I also had some issues with running unit tests locally, again documented below. w.r.t the unit tests, I think the tox environment isn't correctly configured. Additionally, some unit tests failed.

install issue

To pip install the package, I tried what was listed in the README, but I got this error:

pip install https://github.com/pmeier/[email protected]
Collecting https://github.com/pmeier/[email protected]
  ERROR: HTTP error 404 while getting https://github.com/pmeier/[email protected]
ERROR: Could not install requirement https://github.com/pmeier/[email protected] because of HTTP error 404 Client Error: Not Found for url: https://github.com/pmeier/[email protected] for URL https://github.com/pmeier/[email protected]

I instead used pip install git+https://github.com/pmeier/[email protected]

This is my pip version to help debugging the issue:

pip --version
pip 20.2.2 from /home/bluebox/miniconda3/envs/pystiche_review/lib/python3.6/site-packages/pip (python 3.6)

missing documentation

pystiche.meta, pystiche.misc.build_complex_obj_repr, pystiche.ops.functional.value_range_loss, some functions in pystiche.image, most functions in pystiche.image.transforms didn't have any documentation beyond input/return type information

issue with running the example

I had issue running the first example example_nst_with_pystiche.py downloaded from https://pystiche.readthedocs.io/en/stable/_downloads/36c76ed9ccf74ecc12b41424d1ea227a/example_nst_with_pystiche.py which was linked from the end of this page: https://pystiche.readthedocs.io/en/stable/galleries/examples/beginner/example_nst_with_pystiche.html#sphx-glr-galleries-examples-beginner-example-nst-with-pystiche-py

The exact error was:

Traceback (most recent call last):
  File "example_nst_with_pystiche.py", line 116, in <module>
    images.download()
  File "/home/bluebox/miniconda3/envs/pystiche_review/lib/python3.6/site-packages/pystiche/data/collections/download.py", line 200, in download
    image.download(root=root, overwrite=overwrite)
  File "/home/bluebox/miniconda3/envs/pystiche_review/lib/python3.6/site-packages/pystiche/data/collections/download.py", line 144, in download
    raise FileExistsError(msg)
FileExistsError: bird1.jpg with a different MD5 hash already exists in /home/bluebox/.cache/pystiche. If you want to overwrite it, set overwrite=True.

I have never installed pystiche on this machine before. To double-confirm, I removed rm -rf /home/bluebox/.cache/pystiche and reran the script again, but the error is the same.

To help debug:

python --version
Python 3.6.12 :: Anaconda, Inc.

pip list|grep torch
torch             1.6.0
torchvision       0.7.0

pip list|grep pystiche
pystiche          0.5.1

issue with running unit tests

Running tests didn't work out of the box. I was following the instructions in CONTRIBUTING.rst: https://github.com/pmeier/pystiche/blob/v0.5.1/CONTRIBUTING.rst#tests

See the log for details on the environment, commands used, exact output, etc: https://gist.github.com/soumith/70f989dbd9d9d5702cbb24143e88755c

I decided to try get these to work manually by calling pytest . and fixing issues. I had to do the following:

pip install dill
pip install https://github.com/pmeier/pyimagetest/archive/v0.2.zip # i.e. the previous version of pyimagetest
pip install pillow-affine
pip install pytest-subtests

After this, pytest . started working, but one test failed: https://gist.github.com/soumith/690f8fdc9a224f85a0484a04f65d40ec pytest . --skip-slow worked to completion

This is only for manywheel right?

wrt conda, we currently have mkl >= 2018. We might not be able to change it to add the < 2020.1 because that would downgrade other conda packages down, such as numpy/scipy. realized this, so wanted to flag before I forgot.

@brando90 looks like you are on a ppc64le which we don't have binaries for on the default pytorch channel (maybe IBM has them in their ppc64le channel)

Fixes #{issue number}

let's rely on pytorch.org to disseminate this information, as it might get outdated.

apparently we've moved to a once-per-day update of the site now. it should be live by tomorrow morning.

thanks for the great contribution. it should go live sometime today, maybe in a couple of hours.

Please kindly review our Speech-To-Text models

would this require omegaconf and torchaudio to be installed?

if so, you should add a cell like in https://github.com/pytorch/hub/blob/master/nvidia_deeplearningexamples_waveglow.md#example which pip installs these extra packages.

that will make the thing instantly run in Google Colab, and is really valuable!

instead of a zip file, extracting it and then running a batch of wav files through, it would be much nicer and more illustrative of downloading a single wav file and processing it through.

like:

torch.hub.download_url_to_file('some download url for speech.wav', dst='speech.wav)

input = prepare_model_input(glob('midlands_english_female/*.wav'), device=device)

output = model(input)

hey everyone, still working on the review, but I should have something by EOD on the 23rd.

Currently pix_to_3dpt & get_current_pcd function in habitat wrapper returns point in global frame while in locobot, it returns it in robot base frame

Ok, yea then when we were implementing it for Habitat, we completely had the wrong idea.

How does one then transform the point cloud into the global frame after it is returned by get_current_pcd?

is this correct?

@kalyanvasudev said (when we were debugging last time) that PYROBOT frame is basically the global coordinate frame in the pyrobot environment.

The line above this also describes it differently from robot base frame:

the thing to do next is to transform the points from camera frame into the
        global frame of pyrobot environment

thanks so much @sublee , great news.

@peterjc123 would know.

Unnecessary means that it is OK to add the layer if we use D_block way to simplify code?

Yes you can add if you want.

And BTW, in my stupid code, I found pytorch is slower than mxnet. Can you help me improve the pytorch code...

Maybe PyTorch is slower than MXNet :)

hey, wanted to second the excitement.

It's also important to recognize and acknowledge contributions. So, from the PyTorch side, we will commit to adding in the release notes (cc: @pritamdamania87 @mrshenli ) that the code was forked from kakaobrain/torchgpipe and recognize your efforts. If you would like additional ways of acknowledgement, happy to help with that as well.

hey, BatchNorm seems unnecessary for the first layer, as the inputs to the first Conv2d are already zero-mean and unit std, and the Conv2d is initialized (initially in a random draw) in a way that the output of the Conv2d roughly maintains zero-mean and unit std.

This is why we avoided putting batchnorm in the first layer in our original code / paper

hey, BatchNorm seems unnecessary for the first layer, as the inputs to the first Conv2d are already zero-mean and unit std, and the Conv2d is initialized (initially in a random draw) in a way that the output of the Conv2d roughly maintains zero-mean and unit std.

This is why we avoided putting batchnorm in the first layer in our original code / paper

Hi!

Thanks for making this repo public, it's very helpful. I've noticed that PyTorch is pushing pip manywheel nightlies for python 3.8, but it seems like 3.8 is missing from the configuration in cron/ (e.g., 3.8 isn't defined here https://github.com/pytorch/builder/blob/master/cron/build_multiple.sh#L55 , among other places)

Am I missing something, or is the cron configuration to build 3.8 pip manywheels just not pushed yet?

Thanks in advance!

the cron folder is a bit outdated. now the binaries are triggered by CircleCI in this folder: https://github.com/pytorch/pytorch/tree/master/.circleci

Here's 3.8 spec: https://github.com/pytorch/pytorch/blob/master/.circleci/cimodel/data/binary_build_data.py#L36 and https://github.com/pytorch/pytorch/blob/master/.circleci/cimodel/data/dimensions.py#L14

Sorry, that folder is a bit over-engineered

thanks!

Unidecode package was required for loading tacotron2 but it was installed later.

Per title. Layout of pred is now correctly propagated so that correct is not contiguous.

i'll send in my review by that deadline. thanks for the heads-up!

@datametrician i'd let @csarofeen @ptrblck and @ngimel figure out next steps from the PyTorch side. I believe they are figuring out the subtle details and exact plan

Spelling mistake led to errors for beginners.

Reproduce

• Browse to https://pytorch.org/tutorials/intermediate/model_parallel_tutorial.html#apply-model-parallel-to-existing-modules . second line.

cc: @zasdfgbnm @csarofeen

if we can't prune, we can't ship CUDA 11 binaries to PyPI, and I know CuBLAS/CuDNN releases are really long, so worth getting a head-start on this

@malfet we should consider reverting this, in that case.

Basically, MKL 2019 / 2020 / 2020.1 don't really give that much extra perf benefit. The fact that Intel removed this workaround makes us need to evaluate our strategy here, honestly. Should we consider switching to OpenBLAS (and write our own FFT functions)?

cc: @ngimel @jamesr66a @gottbrath

you mean to CUDA 11.0 builds

Adding Ray to the ecosystem

cc @CamiWilliams

@jrhemstad the catch22 is not when a user picks a bad custom allocator. It is when a library that the user is using opaquely picks a bad custom allocator. In that situation, the user might not even be aware that this is happening under the hood, they only see bad performance and attribute it ot "something something pytorch bad..."

this has to be featured_image_2

I'm generally supportive of the direction, but we do need to figure out some details.

What @colesbury said would be technical needs. How it might manifest is in the form of import side-effects.

import foo    # here, foo imports and sets torch to an external allocator that is suboptimal
import torch

out = mymodel(x) # this line runs much slower if `foo` is imported because of the suboptimal overriden allocator

Detecting a suboptimal allocator I think isn't technically possible (other than profiling). So, when an external allocator is enabled, a reasonable thing for pytorch to do is to enable warnings to the user that something suboptimal might happen.

My concern is around whether we have an external memory allocator that we all agree is reasonably performant, that we allow overrides to (or allow overrides to without emitting a warning).

Any thoughts?

no, it was landing, it's done now.

why are we using llvm?

@trevorgribble CUDA Capability 3.0 is not actually supported by PyTorch, as we use __shfl instructions at places, which only came out starting CUDA capability 3.5.

thanks @mattip !

xref pytorch/pytorch#32004

Run github actions CI once a day (at 3:00) and if it fails, open an issue. mattip/pytorch-examples#21 is a forced example of such an issue. A few things to note:

• The main purpose of this action is to consistently test this repo against pytorch nightly builds, to see if some new pytorch "feature" breaks one of the examples.
• Anyone who forks this repo will, by default, get this scheduled action. I added a note to the README. We could also add a step to emit an issue if {{ github.repository_owner != 'pytorch' }} which would hopefully get people's attention :). By default forks do not have issues, so I am not sure what would happen. Forking this repo does seem like a popular thing to do: there are 64.k forks. Searching for a way to disable via the yml led me to this discussion which seems to indicate it cannot be done right now. Another option would be to move from github actions (which are available on all public repos) to travis or circleci, which would require action on part of the fork to enable it. I can pivot this PR to do that, but a admin would have to enable another CI service.
• I hope someone is getting mails on all new issues in this repo, otherwise the issue will not get much attention.

@gchanan @houseroad what does one have to do to get the Backward Compatibility tests to pass? this PR adds a new option at the end with a default, so it technically should be backward compatible.

just checking that this is not Vec(scaling_factor)

dont care as long as it compiles. afaicr devtoolset9 (gcc-9) didn't compile pytorch when we checked a long time ago. it probably does, now.

this is pretty cool!

Add fizz/buzz example, you can use --num-digits to assign the training/testing digit.

thanks for the contribution, but it doesn't seem like a good representative example of anything people would want to do, and hence learn from

thanks!

Minor fix of github CI badge for better rendering.

conda binaries shouldn't have been broken, only the wheels were broken.

-Wl,exclude-libs is a reasonable and easy fix, we should do it.

Presumably, has to be inserted here: https://github.com/pytorch/pytorch/blob/6e4f99b0633f704f8c7bf3bb3b9733680f5dc18b/cmake/public/cuda.cmake#L256-L271

follow along USE_STATIC_CUDNN=1

Adding a section to CONTRIBUTING sounds better, or a PR Template that is extremely simple, for example like literally having one line that says "Fixes ### " so that people don't forget to list the issues it fixes.

this is what we do, instead of using auditwheel: https://github.com/pytorch/builder/blob/master/manywheel/build_common.sh#L231-L323

• we unzip the wheel
• we copy over the dependencies
• we patch the RPATH for these dependencies
• we generated a new RECORD file in the wheel with the new files + hashes
• we zip back the wheel

i'll defer the review to @fmassa

since you are already landing this PR, can you please improve the warning here. Maybe to the effect of:

"With the current PyTorch installation, [device name] with CUDA capability [capability] is not compatible. The current PyTorch install supports CUDA capabilities [arch_list]. If you want to use the [device name] GPU with PyTorch, you might have to compile PyTorch from source"

OR

"With the current PyTorch installation, [device name] with CUDA capability [capability] is not compatible. The current PyTorch install supports CUDA capabilities [arch_list]. If you want to use the [device name] GPU with PyTorch, you might have to download and install the all-GPU PyTorch wheels from the following page [URL]"

I'd rather keep 3.7+PTX than 5.0. 3.7 is used much more than 5.2 / 5.x

Add github action to run examples on CPU

🚀 Feature

Implement a dataloading functionality for reinforcement learning state, action pairs, with assigned policy scores, transitional probabilities and rewards. Implement a set of gradient algorithms (qlearning, actor-critic, etc) in order to populate policy maps and use them or export them for other models.

Motivation

I feel there is a gap in pytorch, which doesn't address a relatively interesting field of machine learning. I understand that reinforcement learning falls outside deep learning, and that simple examples of DQN already exist in tutorials, blogs and such, but I feel that impementing a GPU and CPU based dataloader which is easy to use and understand, combined with the algorithms that calculate the policy values and maps, would benefit pytorch as well as those who already use pytorch for a variety of reasons. I think that there's a lot of software domains where this would be applicable.

Pitch

1. develop a dataloader class or set of classes which deal with deterministic state-action spaces and with probabilistic state-action spaces.
2. develop a set of reinforcement learning algorithm classes which can be applied to those dataloaders, and populate policy values, based on rewards and such

Alternatives

No much really.

Additional context

I think that pytorch has grown to be a wonderful tool used in so many different places, and that reinforcement learning seems to be missing. I know theres a tutorial for DQN, and I know that a lot of the algorithms are trivial and simple, yet it seems to me that users rely on pytorch as a simple and easy to use tool for machine learning, and as such don't always have the time (or know-how) on how to implement that functionality.

Just to be clear I propose I implement this with some guidance, as I've already done this in C, C++ and Python before. I've got a few ideas on how to do this with the underlying C++ aten and autograd libraries and then move on to the python bindings.

cc @SsnL

Tweak the run_python_examples.sh script to complete in ~5 minutes (on my machine) without CUDA. Once this is merged, we can consider adding #704 to actually run the examples as part of CI.

TBD: how do we get a pre-release version of torch/torchvision in order to test the latest HEAD? Perhaps it should be a pytorch/pytorch cron job that runs once a day or once a week?

ping @jlin27

I've taken a look at the PR, sorry for the delay.

It looks to be in a reasonably good shape, and covers my original outline of requirements pretty well.

At first, I also thought about creating a LazyModule and LazyLinear classes, but since that conflicted with @soumith proposal I went to directly edit the modules themselves. However, now that we have to throw #29523 into the mix maybe it is the best option.

Generally, my point is about explicitness -- the user has to be very aware that they are doing this shape inference scheme, which LazyLinear does cover. In fact LazyLinear() might be easier to write, than nn.Linear(nn.INFER, ...).