rusty1s/pytorch_geometric 9206

Geometric Deep Learning Extension Library for PyTorch

PyTorch Extension Library of Optimized Scatter Operations

PyTorch Extension Library of Optimized Autograd Sparse Matrix Operations

PyTorch Extension Library of Optimized Graph Cluster Algorithms

Generative Adversarial Networks and Autoencoders for 3D Shapes

rusty1s/deep-graph-matching-consensus 112

Implementation of Deep Graph Matching Consensus in PyTorch

Source code for our AAAI paper "Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks".

rusty1s/pytorch_spline_conv 84

Implementation of the Spline-Based Convolution Operator of SplineCNN in PyTorch

ES6 RESTFul Koa2 API with Mongoose and OAuth2

Embedded Graph Convolutional Neural Networks (EGCNN) in TensorFlow

issue commentrusty1s/pytorch_geometric

torch.jit.frontend.NotSupportedError when creating jittable GNN operators（message passing）

I identified two issues:

- You cannot pass
`data`

to the GNN`forward`

since`data`

is not jittable. You need to pass the used tensors individually,*e.g.*:

```
def forward(self, x: Tensor, edge_index: Tensor):
...
```

- It looks like
`Sequential`

makes some problems here. The code runs through if I replace`Sequential`

with`ModuleList`

,*e.g.*:

```
self.layer_seq = nn.ModuleList()
for i in range(num_subgraph_layres):
t1='(Tensor,Tensor)->Tensor'
self.layer_seq.add_module(
GraphLayerProp(in_channels, hidden_unit).jittable(t1))
for layer in self.layer_seq:
x = layer(x, edge_index)
```

comment created time in 15 hours

push eventrusty1s/pytorch_geometric

commit sha 6304c7c29883cf2aaae0e8c0ffcd12c58dcbb7ed

an gat sampling example using multi-gpu training

commit sha b3ad9b8a1d5080c523314d521c34f85a74d6a496

clean up

commit sha 23a0996d40c4a4d2d519b762b2002eb406e24303

Clean up 2

commit sha caf5f57bf10f9b697b418ea7ec50594ee7a21b73

Merge pull request #1453 from maqy1995/pytorch_geometric_maqy gat sampling example using multi-gpu training

push time in 18 hours

PR merged rusty1s/pytorch_geometric

pr closed time in 18 hours

pull request commentrusty1s/pytorch_geometric

gat sampling example using multi-gpu training

Awesome :) Sorry that it took me so long to merge.

comment created time in 18 hours

push eventmaqy1995/pytorch_geometric

commit sha 23a0996d40c4a4d2d519b762b2002eb406e24303

Clean up 2

push time in 18 hours

push eventmaqy1995/pytorch_geometric

commit sha b3ad9b8a1d5080c523314d521c34f85a74d6a496

clean up

push time in 18 hours

issue commentrusty1s/pytorch_geometric

torch.jit.frontend.NotSupportedError when creating jittable GNN operators（message passing）

Can you show me an example to reproduce?

comment created time in 21 hours

issue commentrusty1s/pytorch_geometric

How to implement other GraphSAGE Aggregator functions like Max Pool?

You can install from source:

```
pip install git+https://github.com/rusty1s/pytorch_geometric.git
```

or change the `aggr`

flag after initialization:

```
conv = SAGEConv(...)
conv.aggr = 'max'
```

comment created time in 21 hours

issue commentrusty1s/pytorch_geometric

PyTorch Geometric binaries for PPC

Sounds interesting. We currently build our binaries using TravisCI, which looks like to support PPC architectures.

comment created time in 21 hours

issue commentrusty1s/pytorch_geometric

att_r Parameter not updating in GATConv

Thanks for this issue. This is already fixed in master, see here.

comment created time in 21 hours

issue commentrusty1s/pytorch_geometric

Add properties to Dataset object

Hi and thanks for this issue. However, I'm not sure if I understand the problem exactly. What's wrong with your current "workaround"? Note that you are free to save anything you like inside your custom dataset. For example, we already do this for the `GEDDataset`

, see here.

comment created time in 21 hours

issue commentrusty1s/pytorch_geometric

Can I have an example to use GNNExplainer for link prediction?

Sure, that can work. However, note that during inference, `GraphSAGE`

operates on the full graph with `NeighborSampler`

size `=-1`

, meaning that you can use a single `edge_mask`

for consecutive layers.

comment created time in 21 hours

issue commentrusty1s/pytorch_geometric

one error when I run function topk() on the last batch

You can use

```
A = SparseTensor.from_edge_index(edge_index, edge_attr, sparse_sizes=(N, N))
```

comment created time in a day

issue commentrusty1s/pytorch_geometric

Yes, you can do that. Alternatively, you can use `torch_geometric.data.DataLoader`

to train in mini-batches:

```
loader = DataLoader([cora] * N, batch_size=...)
for data in loader:
mse_loss(...)
```

comment created time in a day

push eventrusty1s/pytorch_geometric

commit sha a91892f5ab03256d4de1f3b1a73090cf5d704e8a

added infer block to infer link_prediction.

commit sha d2acf3a474d3174855aaa78c2f77ebaea1f190a1

forward_all usage

commit sha ef1d82b706387ba9f91e7a55d0209518b4224328

clean up

commit sha 35e9eced32271de4e1ab805c1e223bd95123a0ce

Merge pull request #1733 from reshinthadithyan/master Addition of Inference Block in examples/link_pred.py

push time in 2 days

PR merged rusty1s/pytorch_geometric

The existing version of examples/link_pred.py doesn't have a proper inference example. Refer #1699, added a inference block with `InnerProductDecoder.forward_all()`

to give out a probability adjacency matrix. This would give a cleaner view on usage in inference stage.

pr closed time in 2 days

pull request commentrusty1s/pytorch_geometric

Addition of Inference Block in examples/link_pred.py

Thanks a lot! I modified it a bit. In particular, I moved the encoding and decoding part to the model.

comment created time in 2 days

push eventreshinthadithyan/pytorch_geometric

commit sha ef1d82b706387ba9f91e7a55d0209518b4224328

clean up

push time in 2 days

push eventrusty1s/pytorch_geometric

MohammadAmin Khashkhashi Moghaddam

commit sha f5f200759407806c19f00b46919de3ff6eed4385

Accept custom aggregate method for convolution layers Set the aggr method only if it is not already set by the user

commit sha d0d552ce065ca46f42043fa83cb5e1f355ce8156

Merge pull request #1717 from m30m/master Accept custom aggregate method for convolution layers

push time in 2 days

PR merged rusty1s/pytorch_geometric

Set the aggr method only if it is not already set by the user

pr closed time in 2 days

pull request commentrusty1s/pytorch_geometric

Accept custom aggregate method for convolution layers

I see. That makes sense, thanks a lot :)

comment created time in 2 days

issue commentrusty1s/pytorch_scatter

This might be relevant: https://github.com/rusty1s/pytorch_geometric/issues/1593

comment created time in 2 days

issue commentrusty1s/pytorch_geometric

Yes, Eq (7) looks wrong from a dimensionality perspective. You can either append zero vectors to your central node features or use a linear projection to transform incoming messages to the same dimensionality.

IMO, you already learn the weights per feature since `gate_x_j`

computes a weight for each feature.

comment created time in 2 days

push eventsnap-stanford/ogb

commit sha 8548e78b7f74e8cc491246d2833b4e046ebe3110

add dynamic collab experiment

commit sha 6e1424114e985dd19bad800ab7dab53a2852c267

Merge branch 'master' of github.com:snap-stanford/ogb

push time in 2 days

issue commentrusty1s/pytorch_geometric

OSError: [WinError 127] The specified procedure could not be found

This is a weird PyTorch error, and I already encountered it by myself. There are also a lot of users reporting similar problems. I personally fixed it by just setting up a fresh miniconda environment.

comment created time in 2 days

issue commentrusty1s/pytorch_geometric

Can I use GCN Conv to a weighted graph?

Yes, you can simply pass in the `edge_weight`

to `GCNConv.forward`

as an additional attribute.

comment created time in 2 days

issue commentrusty1s/pytorch_geometric

Can you do me a favor and try out `SAGEConv`

instead of `GCNConv`

and see if that succeeds?

comment created time in 2 days

issue commentrusty1s/pytorch_geometric

Not found error for torch_sparse::ptr2ind in torchscript

Thanks, that was indeed a typo, it is fixed now :) I also added this line to the CMake file, and hope that is solves your issues.

comment created time in 2 days

push eventrusty1s/pytorch_sparse

commit sha a1021ebd018cd473e49c66a294e166f834e96c8b

fix

push time in 2 days

issue commentrusty1s/pytorch_geometric

Can I have an example to use GNNExplainer for link prediction?

In general, `GNNExplainer`

only learns a single edge mask, and thus `GNNExplainer`

is not trivial to apply in a `NeighborSampler`

scenario. Since `GNNExplainer`

only operates on the `L`

-hop neighborhood around a single node, I do not see the need to apply neighbor sampling here. As an alternative, I suggest to define a model `forward`

function, that operates on a single `edge_index`

and use that one for explaining nodes/links, *e.g.*:

```
def forward(self, x, edge_index):
...
def forward_with_sampling(self, x, adjs, link, n_id):
...
```

comment created time in 2 days

issue commentrusty1s/pytorch_geometric

one error when I run function topk() on the last batch

It's hard for me to say what went wrong solely based on the error message. Can you show me a minimal example to reproduce?

comment created time in 2 days

issue commentrusty1s/pytorch_geometric

Your first operator looks correct to me, but `emb_dim`

should not be static (and actually depends on the output dimension of `self.nn`

of the previous layer).

comment created time in 2 days

issue commentrusty1s/pytorch_geometric

Can you specify what you are trying to achieve?

comment created time in 2 days

push eventrusty1s/pytorch_geometric

commit sha 4b2d17e639d631984997bde57ce8a562b28cc33a

fix docs

push time in 2 days

push eventrusty1s/rusty1s.github.io

commit sha fc684bbd893bb9ad31d4e64413f58a353fb02021

update

push time in 2 days

push eventrusty1s/pytorch_geometric

commit sha b88e3823ba18c70485a5e232aed6bae89bb0cb21

don't read node features on g-plus since they are not consistent

push time in 3 days

push eventrusty1s/pytorch_geometric

commit sha ea3577a15d2eaacdcbb9f0aeeb97f6630ac207cf

fix empty edges

push time in 3 days

issue commentrusty1s/pytorch_geometric

Should be working now :) Sorry for the inconveniences!

comment created time in 4 days

push eventrusty1s/pytorch_geometric

commit sha 708e7e41b72a5bb08c05ba1fab38c387796aed66

fix gplus

push time in 4 days

issue commentrusty1s/himp-gnn

adding tests on regression tasks

That should be straigtforward to implement by training against ground-truth targets with `F.mse_loss`

.

comment created time in 4 days

issue closedrusty1s/himp-gnn

Using PyG 1.5.0 or 1.6.1

got this error during python run:

Traceback (most recent call last): File "train_tox21.py", line 113, in <module> if val_perf > best_val_perf: TypeError: '>' not supported between instances of 'dict' and 'int'

same for Hiv:

Traceback (most recent call last): File "train_hiv.py", line 113, in <module> if val_perf > best_val_perf: TypeError: '>' not supported between instances of 'dict' and 'int'

closed time in 4 days

thegodoneissue commentrusty1s/himp-gnn

Thanks, that's indeed a bug. Fixed it.

comment created time in 4 days

push eventrusty1s/himp-gnn

commit sha 5f8573b1a6f579ae95c918bbd2a44dacf0db2e37

fix test

push time in 4 days

issue commentrusty1s/pytorch_geometric

How can I implement A times X？

Sure:

```
from torch_sparse import SparseTensor
A = SparseTensor.from_dense(torch.randn(100, 100))
X = torch.randn(100, 32)
out = A @ X
```

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

That looks correct to me.

comment created time in 4 days

push eventrusty1s/pytorch_geometric

commit sha 0f5b422ce04039b97a9d92b0d93109200d5d650c

linting

push time in 4 days

issue commentrusty1s/pytorch_geometric

GNNExplainer for graph classification problems

Of course, you can use a global pooling layer. I was talking about the hierarchical pooling layer implemented via `graclus`

and `max_pool`

.

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

Concatenation as aggregation function?

Yes, given that our edge indices are sorted based on destination nodes, you can define a custom aggregation function via `aggregate`

, *e.g.*:

```
def aggregate(self, inputs): # [num_edges, num_features]
# Reshape to [num_nodes, k, num_features]
inputs = inputs.view(-1, k, inputs.size(-1))
...
```

comment created time in 4 days

push eventrusty1s/pytorch_geometric

commit sha bee454aaeeddf5af09532189e3d129577b00dae2

use indices in shortest_path

commit sha 4cf349fc2af4bcfbcbce6fd14667d571733581f1

Merge pull request #1731 from muhanzhang/master Use indices in shortest_path() in seal_link_pred.py

push time in 4 days

PR merged rusty1s/pytorch_geometric

Previously the **shortest_path()** calculates shortest path distances between all pairwise nodes, but we only need the shortest path distances from all nodes to the src and dst nodes. To avoid redundant computation, I specify *indices* in **shortest_path()** directly. This brings 3 times speed up on Cora.

pr closed time in 4 days

pull request commentrusty1s/pytorch_geometric

Use indices in shortest_path() in seal_link_pred.py

Thanks :)

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

GNNExplainer for graph classification problems

The problem is that the pooling approach in the MNIST example is non-deterministic, and can produce a different amount of edges between different runs. Therefore, shapes between edge masks and actual used edges do not longer match. I suggest you try out a model without pooling and see if that succeeds :)

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

You would need to implement the `PointConv`

via `scatter_max`

from the `torch-scatter`

package. Everything else can be implemented via `torchlib`

and `torch-cluster`

.

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

That is indeed compatible, and can be implemented in the `message`

function via `torch.cat`

, *e.g.*:

```
def message(self, h_j, z_j):
return torch.cat([h_j, z_j], dim=-1)
```

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

Can I have an example to use GNNExplainer for link prediction?

I think that the `GNNExplainer`

can be modified to the task of link prediction. You may want to add a new function `explain_link`

to achieve that. Note that at the moment, this is restricted to full-batch subgraphs, and cannot handle multiple adjacencies.

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

Could I create 2 different batches in data

You can pass the `follow_batch`

argument to the `DataLoader, *e.g.*:

```
loader = DataLoader(dataset, batch_size, follow_batch=['posx']
```

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

Questions about the GraphSAGE example

You can also sample the complete 2-hop neighborhood, but this is rather expensive (especially for Reddit). We apply here trick here to compute the node embeddings efficiently: Instead of sampling multiple layers for a mini-batch, we instead compute the node embeddings layer-wise. Doing this exactly two times mimics a two-layer GNN.

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

Your code looks correct to me. Can you apply the suggestion and see which op fails to compute gradients?

```
Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
```

comment created time in 4 days

issue commentrusty1s/pytorch_geometric

Not found error for torch_sparse::ptr2ind in torchscript

I added a header file to `torch-sparse`

. Please try again :)

comment created time in 4 days

push eventrusty1s/pytorch_sparse

commit sha 2bcc13ed3dd2cac1a023601a622307b74efa4db8

added header file

push time in 4 days

issue commentrusty1s/pytorch_geometric

Citeseer dataset isolated nodes

I'm not sure what you are trying to do. What is the exact problem with reading `citeseer.cites`

and `citeseer.content`

?
If you want to run `tkipf/pygcn`

with CiteSeer, it's also maybe a good idea to ask for help there.

comment created time in 4 days

push eventrusty1s/pytorch_geometric

commit sha 66ff7e880b720fc6f4b707360bdf3d19873c3d79

reset numba version

push time in 4 days

issue commentrusty1s/pytorch_geometric

Dataset not found [Reddit and PPI]

Mh, that's interesting, and I believe it has something do to with `numba`

. Can you please try again?

comment created time in 4 days

issue closedrusty1s/pytorch_geometric

## 🐛 Bug

using `torch_geometric.datasets.SNAPDataset`

with dataset name "ego-twitter" and "ego-gplus" results in an error.

## To Reproduce

Steps to reproduce the behavior:

```
import torch_geometric.datasets as DS
data = DS.SNAPDataset("data/",name="ego-gplus")
```

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

e.g.: the "ego-twitter" configuration would lead to:

```
KeyError: 83943787
```

## Environment

- OS: Linux
- Python version: 3.8.3
- PyTorch version: 1.6.0
- CUDA/cuDNN version: 10.1
- GCC version:
- Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->

closed time in 7 days

Magolorissue commentrusty1s/pytorch_geometric

Fixed in master, see https://github.com/rusty1s/pytorch_geometric/commit/c56a50eb2f17a41c4b52d837eb66a44d8efbfee5. Thanks for reporting!

comment created time in 7 days

issue closedrusty1s/pytorch_geometric

Dataset not found [Reddit and PPI]

Hi, The dataset for cluster examples (cluster_gcn_reddit.py and cluster_gcn_PPI.py) throws "HTTP Error 404: Not Found" For reddit, it tries to download from Downloading https://s3.us-east-2.amazonaws.com/dgl.ai/dataset/reddit.zip Has the dataset moved, or I am doing something wrong? Thanks.

closed time in 7 days

snash4push eventrusty1s/pytorch_geometric

commit sha c56a50eb2f17a41c4b52d837eb66a44d8efbfee5

fix idx numerical issues

push time in 7 days

issue commentrusty1s/pytorch_geometric

Working with large graph for node prediction!

Since your graphs are rather dense with a rather low number of nodes, I suggest you look into the newly introduced `SparseTensor`

format, see here.

If you want to make use of mini-batch training, we provide several options:

comment created time in 7 days

issue commentrusty1s/pytorch_geometric

There are no C++ examples of PyTorch Geometric. PyG is purely Python-based. Nonetheless, with release 1.6.0, we support TorchScript, which should help you to run your models in non-Python environments. You can find some basic examples here and a short tutorial here.

comment created time in 7 days

issue commentrusty1s/pytorch_geometric

Citeseer dataset isolated nodes

You can find the same pre-processing technique as in `tkipf/gcn`

here.

comment created time in 7 days

issue commentrusty1s/pytorch_geometric

Not found error for torch_sparse::ptr2ind in torchscript

That looks good to me. Does that work for you?

comment created time in 7 days

issue commentrusty1s/pytorch_geometric

GEDDataset num_classes request leads to AttributeError

You can access them via `data.x`

.

comment created time in 7 days

issue commentrusty1s/pytorch_geometric

How to get value pass through using knn?

That's correct. `knn`

computation has now backward computation since its output is discrete. I'm not sure what you are trying do to, but this should not be the reason for any errors you are receiving during back-propagation.

comment created time in 7 days

issue commentrusty1s/pytorch_geometric

Can you show me your model to reproduce this?

comment created time in 7 days

push eventrusty1s/pytorch_sparse

commit sha a9d4d4695fb58aa0a2696443ab8f9dd1d068c19c

typo

push time in 7 days

issue commentrusty1s/pytorch_geometric

Metis [segmentation Fault on ClusterData ]

Can you try to make your graph undirected and see if that fixes the issue?

comment created time in 7 days

push eventrusty1s/pytorch_geometric

commit sha 68ca38e28c1b86cf267fab8499c5f3e52d8ec410

fix broken urls

push time in 7 days

issue commentrusty1s/pytorch_geometric

Dataset not found [Reddit and PPI]

This is now fixed in master. You can simply re-install via:

```
pip install git+https://github.com/rusty1s/pytorch_geometric.git
```

Thanks for reporting!

comment created time in 7 days

issue commentrusty1s/pytorch_sparse

No, wheels are not uploaded to PyPI since different CUDA versions require different wheels (which is not covered by PyPI).

comment created time in 8 days

issue commentrusty1s/pytorch_geometric

[Roadmap] PyTorch Geometric 1.6.1 and 1.7.0

cugraph conversions is a good idea, I added it to the roadmap. The SuperpixelDataset is already fixed in master.

comment created time in 8 days

issue commentrusty1s/pytorch_geometric

Not found error for torch_sparse::ptr2ind in torchscript

For executing in a C++ environment, please ensure that you also have the C++ extensions installed, see here.

comment created time in 8 days

issue commentrusty1s/pytorch_geometric

Seems to be a bug. I will look into it.

comment created time in 8 days

pull request commentrusty1s/pytorch_geometric

Accept custom aggregate method for convolution layers

Thanks for the PR. While I agree that the `aggr`

keyword is missing for a few operators, I feel like allowing the user to modify the aggregation type for any kind of operator is not particular useful, *e.g.*, for `GCNConv`

or `GATConv`

which are defined to perform weighted mean normalization.

comment created time in 8 days

issue commentrusty1s/pytorch_geometric

Understanding examples/link_pred.py

You can use the `Decoder`

to reconstruct such probabilistic adjacency matrix.

comment created time in 8 days

issue commentrusty1s/pytorch_geometric

The idea would be to "scatter" pixel feature information of a superpixel into a single representation. Consider that you have a "mask" `idx`

holding indices to which superpixel a pixel belongs. Then you can do the following to train your model in an end-to-end fashion:

```
idx = ... # [height, width] holding indices from 0 to N - 1 superixels
img = ... # [height, width, num_features]
node_x = scatter(img.view(-1, num_features), idx.view(-1), dim=0, dim_size=N, reduce="mean")
```

comment created time in 8 days

issue commentrusty1s/pytorch_geometric

My guess is that, data objects of different graphs may have different feature dimensionalities, *e.g.*, `data1.x.size(-1) == data2.x.size(-1)`

and `data1.edge_attr.size(-1) == data2.edge_attr.size(-1)`

. Can you verify that this is the case?

comment created time in 8 days

issue commentrusty1s/pytorch_geometric

Message passing with more than two nodes

Are you referring to hypergraphs? This is currently not really covered by the `MessagePassing`

interface, so you may need to write logic for that by yourself.

comment created time in 8 days

push eventrusty1s/pytorch_geometric

commit sha e3fb65d2703fe5ca073a255d956e41364f8d89ef

Change accuracy return type to float

commit sha ffa52fc77e622044d96282ee3e28dc9776bb39d0

Merge pull request #1712 from LukasZahradnik/master Change accuracy return type to float

push time in 10 days

PR merged rusty1s/pytorch_geometric

While going through the documentation, I noticed that `torch_geometric.utils.accuracy`

has wrong return type (`int`

).

pr closed time in 10 days

push eventrusty1s/pytorch_geometric

commit sha ee17659ebc16d17e778c00bc1195d386b44cdff4

Fix typo in total graphs in ZINC subset

commit sha 50af85b0e8fc4741fa09ebdc260b7e586666b723

Merge pull request #1711 from vijaydwivedi75/master Fix typo in total graphs in ZINC subset

push time in 11 days

PR merged rusty1s/pytorch_geometric

The ZINC subset used in Benchmarking GNNs has 12,000 molecular graphs - 10,000 train/1,000 val/1,000 test.

PR for this minor correction.

pr closed time in 11 days

pull request commentrusty1s/pytorch_geometric

Fix typo in total graphs in ZINC subset

Yeah, thanks :)

comment created time in 11 days

push eventrusty1s/pytorch_geometric

commit sha 9faf964400d08d9345548d95e183551dfb38ec1c

typo

push time in 12 days

push eventrusty1s/pytorch_geometric

commit sha f7c0591007fdf8cf7faac16c0b865900318dde0a

Update README.md

commit sha b2429c0c7d0d3900f74f98b81e42773d972f911e

Update autoencoder.py

commit sha 3bd9fde4c63e252bc8e967515eca2f1664aca6fe

Update autoencoder.py

commit sha f11eabc65072d34b431cc85419c27c0b0d00950c

clean up

commit sha fb1d3f989688e4ca6989350dc58638f574d98e52

Merge pull request #1656 from GuillaumeSalha/linear_gae Linear Graph AE and VAE

push time in 12 days

PR merged rusty1s/pytorch_geometric

Dear @rusty1s,

Once again, congratulations to you and your collaborators for this amazing work!

I am the lead author of the article Keep It Simple: Graph Autoencoders Without Graph Convolutional Networks, presented at the NeurIPS 2019 workshop on Graph Representation Learning. In this article, as well as in an extended conference version recently accepted at ECML-PKDD 2020, we studied simpler versions of Graph Autoencoders (AE) and Graph Variational Autoencoders (VAE), referred to as **Linear Graph AE and VAE**.

In a nutshell, we replaced the multi-layer GCN encoders from standard models (Kipf and Welling 2016) by simpler *linear models* w.r.t. the adjacency matrix of the graph (`Z = AW`

, or `Z = AXW`

in case of node-level features `X`

), involving a unique weight matrix to tune and no activation function. We showed that they often appear as effective and simpler alternatives to GCN-based models, for various graphs including the widely used Cora, Citeseer and Pubmed citation networks, achieving similar or even better performances on downstream tasks such as link prediction or node clustering. In our work, we also tried to provide insights on *when* to use multi-layer GCN encoders and when, on the contrary, these encoders might actually be unnecessarily complex.

We released a TensorFlow implementation of Linear Graph AE and VAE a few months ago. However, some users recently asked for a PyTorch one, and I think that your `pytorch-geometric`

project is the best place to include it.

Therefore, I am opening this PR to propose a modification of `examples/autoencoder.py`

, aiming at including a working example of linear graph AE and VAE with application to link prediction, building upon the existing GAE and VGAE classes. With the proposed implementation, I manage to reach better results than standard GAE/VGAE on the provided Cora, Citeseer and Pubmed datasets from the example file, while being faster. So, I sincerely think this can be of interest for some future users. I also updated the README file accordingly.

Best,

Guillaume

pr closed time in 12 days

pull request commentrusty1s/pytorch_geometric

Thank you :)

comment created time in 12 days

push eventGuillaumeSalha/pytorch_geometric

commit sha f11eabc65072d34b431cc85419c27c0b0d00950c

clean up

push time in 12 days

push eventrusty1s/pytorch_geometric

commit sha 26866fcc3516a17ee2f38504afd9e82901c1d287

Update softmax.py I propose a new version of softmax function in Utils module. The version can reduce a little running time by 3 times, ^ _ ^. ```python from torch import Tensor from torch_scatter import scatter import torch from time import time # set N = 4000 src = torch.rand(N*(N+1)//2) index = [] for i in range(N): tmp = [i]*(i+1) index = index+tmp index = torch.tensor(index,dtype=torch.long ) # norm t1 = time() out = src out = out.exp() out_sum = scatter(out, index, dim=0, dim_size=N, reduce='sum')[index] t2 = time() # local max local_out = src - scatter(src, index, dim=0, dim_size=N, reduce='max')[index] local_out = local_out.exp() local_out_sum = scatter(local_out, index, dim=0, dim_size=N, reduce='sum')[index] t3 = time() # global max global_out = src - src.max() global_out = global_out.exp() global_out_sum = scatter(global_out, index, dim=0, dim_size=N, reduce='sum')[index] t4 = time() print('norm', t2 - t1) print('local softmax', t3 - t2) print('global softmax', t4 - t3) a = out / (out_sum + 1e-16) b = local_out / (local_out_sum + 1e-16) c = global_out / (global_out_sum + 1e-16) eps = 0.0000001 print((torch.abs(a-c)>eps).sum()) print((torch.abs(a-b)>eps).sum()) # ============================================================================= # result # norm 0.04488039016723633 # local softmax 0.1874983310699463 # global softmax 0.06482648849487305 # tensor(0) # tensor(0) # ============================================================================= ```

commit sha 910817862dd01c539e6ad32930fe3d528f5487d2

clean up

commit sha 421fd37ac54c0220134e0f80b29ea7038982ca7c

fix test

commit sha b59619a7980da7edcc5d74f0065fe3b26ecc543c

Merge pull request #1707 from Book1996/patch-1 Update softmax.py

push time in 12 days

PR merged rusty1s/pytorch_geometric

I propose a new version of softmax function in Utils module. The version can reduce a little running time by 3 times, ^ _ ^.

```
from torch import Tensor
from torch_scatter import scatter
import torch
from time import time
# set
N = 4000
src = torch.rand(N*(N+1)//2)
index = []
for i in range(N):
tmp = [i]*(i+1)
index = index+tmp
index = torch.tensor(index,dtype=torch.long )
# norm
t1 = time()
out = src
out = out.exp()
out_sum = scatter(out, index, dim=0, dim_size=N, reduce='sum')[index]
t2 = time()
# local max
local_out = src - scatter(src, index, dim=0, dim_size=N, reduce='max')[index]
local_out = local_out.exp()
local_out_sum = scatter(local_out, index, dim=0, dim_size=N, reduce='sum')[index]
t3 = time()
# global max
global_out = src - src.max()
global_out = global_out.exp()
global_out_sum = scatter(global_out, index, dim=0, dim_size=N, reduce='sum')[index]
t4 = time()
print('norm', t2 - t1)
print('local softmax', t3 - t2)
print('global softmax', t4 - t3)
a = out / (out_sum + 1e-16)
b = local_out / (local_out_sum + 1e-16)
c = global_out / (global_out_sum + 1e-16)
eps = 0.0000001
print((torch.abs(a-c)>eps).sum())
print((torch.abs(a-b)>eps).sum())
# =============================================================================
# result
# norm 0.04488039016723633
# local softmax 0.1874983310699463
# global softmax 0.06482648849487305
# tensor(0)
# tensor(0)
# =============================================================================
```

pr closed time in 12 days