profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/bentrevett/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

bentrevett/a-tour-of-pytorch-optimizers 27

A tour of different optimization algorithms in PyTorch.

bentrevett/code2vec 20

A PyTorch implementation of `code2vec: Learning Distributed Representations of Code` (Alon et al., 2018)

bentrevett/extreme-summarization-of-source-code 11

Implementation of 'A Convolutional Attention Network for Extreme Summarization of Source Code' in PyTorch using TorchText

bentrevett/bag-of-tricks-for-efficient-text-classification 7

Implementation of 'Bag of Tricks for Efficient Text Classification' in PyTorch using TorchText

bentrevett/character-aware-neural-language-models 2

Implementation of 'Character-Aware Neural Language Models' in PyTorch using TorchText

bentrevett/Glucoduino 2

Project to read data from glucometers using the Arduino platform

bentrevett/Glucoduino-Classic-Bluetooth-Application 1

Android application for glucoduino project using standard Bluetooth

bentrevett/Glucoduino-CSR-Chip 1

Code for the CSR uEnergy SDK for the glucoduino project

issue commentbentrevett/pytorch-sentiment-analysis

Using a target size (torch.Size([64, 1])) that is different to the input size (torch.Size([304800, 1])) is deprecated. Please ensure they have the same size

Which line is giving you that error? Tensor shape mismatching is a common bug in deep learning models, and the best way to solve it by making sure the shapes output by each layer is what you expect by printing out the tensor shapes out by each layer and seeing if it matches with what you expect it to be.

KeerthiKrishna97

comment created time in 4 hours

issue commentbentrevett/pytorch-sentiment-analysis

Using a target size (torch.Size([64, 1])) that is different to the input size (torch.Size([304800, 1])) is deprecated. Please ensure they have the same size

Your error is in the line nn.Linear(1024, batch_size). I believe this should be nn.Linear(1024, 8*8) as the generator is trying to output 8x8 images.

Hello. Here 8*8 = 64 is the batch_size

Right, but nn.Linear(1024, batch_size) means it takes in a [batch size, 1024] tensor and outputs a [batch size, batch size] tensor, when it should be a [batch size, channels * height * width] tensor and then the last dimension should be reshaped.

KeerthiKrishna97

comment created time in 4 hours

issue commentbentrevett/pytorch-sentiment-analysis

Using a target size (torch.Size([64, 1])) that is different to the input size (torch.Size([304800, 1])) is deprecated. Please ensure they have the same size

Also, the line x = x.view(-1, batch_size) is an error. It should be x = x.view(-1, batch_size).

KeerthiKrishna97

comment created time in 4 hours

issue commentbentrevett/pytorch-sentiment-analysis

Using a target size (torch.Size([64, 1])) that is different to the input size (torch.Size([304800, 1])) is deprecated. Please ensure they have the same size

Your error is in the line nn.Linear(1024, batch_size).

I believe this should be nn.Linear(1024, 8*8) as the generator is trying to output 8x8 images.

KeerthiKrishna97

comment created time in 4 hours

push eventbentrevett/machine-learning-courses

Ben Trevett

commit sha c11357ff12f0bdde83d552fdff2b08ae73b04ad2

Update README.md

view details

push time in 8 days

issue commentbentrevett/pytorch-seq2seq

Tutorial 1: Differences between Encoder/Decoder in Seq2Seq Model

The embedding layer you don't need to do anything with, however for RNN models, such as the LSTM, if you want the batch dimension first then you need to initialize them with batch_first=True.

michael-camilleri

comment created time in 11 days

push eventbentrevett/notes

Ben Trevett

commit sha ea6e5a7b94ad96663215dc3d5eeec66527a125da

update

view details

push time in 12 days

push eventbentrevett/notes

bentrevett

commit sha ba1a80c7b2ab7d148602c4ec7048154bd6f539b0

update

view details

push time in 18 days

push eventbentrevett/notes

bentrevett

commit sha e8f2fd776d394d422ae40a57463af9957b535be0

update

view details

bentrevett

commit sha bab2c0c086651c90baef68bbe4453f7663d9d975

Merge branch 'master' of https://github.com/bentrevett/notes

view details

push time in a month

push eventbentrevett/machine-learning-courses

Ben Trevett

commit sha 794f049bc8aa04a17b1f09b026b4a515bd8a0610

Update README.md

view details

push time in a month

push eventeasynlp/easynlp

Ben Trevett

commit sha 8d5669597ad16612179e62e854badbb541ce5649

add benchmarks - added benchmark for summarization - added benchmark for question answering

view details

Ben Trevett

commit sha 4b3b405a64ca166cc19ee9c43b79a475cf699996

Merge pull request #5 from easynlp/benchmarks add benchmarks

view details

push time in a month

PR merged easynlp/easynlp

add benchmarks
  • added benchmark for summarization
  • added benchmark for question answering
+29 -0

0 comment

2 changed files

bentrevett

pr closed time in a month

PR opened easynlp/easynlp

add benchmarks
  • added benchmark for summarization
  • added benchmark for question answering
+29 -0

0 comment

2 changed files

pr created time in a month

push eventeasynlp/easynlp

Ben Trevett

commit sha 8d5669597ad16612179e62e854badbb541ce5649

add benchmarks - added benchmark for summarization - added benchmark for question answering

view details

push time in a month

create barncheasynlp/easynlp

branch : benchmarks

created branch time in a month

push eventCodedotAl/reading-group

Ben Trevett

commit sha b5203a449cc87cefebcbe443cf7bcf794826327f

Update README.md

view details

push time in a month

create barnchCodedotAl/reading-group

branch : main

created branch time in a month

created repositoryCodedotAl/reading-group

Information about the CodedotAI reading group sessions.

created time in a month

issue commentbentrevett/pytorch-sentiment-analysis

for word embedding in RNN model

Yes, if you copy the weights from pre-trained embeddings, then they are fine-tuned. The parameters will update as they are not frozen. If you want to freeze the embedding, then you can set embedding_layer.weight.requires_grad = False.

If using Adam, which we do in the tutorials, then weights initialized to zero will not stay zero due to the beta terms. The weight for the padding token will however stay zero as we pass the padding index to the padding_idx argument of the nn.Embedding layer, which explicitly prevents it from changing.

Dongximing

comment created time in a month

push eventeasynlp/easynlp

bentrevett

commit sha 9aa3b21bcab02635796c15a12b1b3ac9a7e42e6c

fix type hints

view details

push time in 2 months

push eventeasynlp/easynlp

bentrevett

commit sha 612d3584ffd88b1158fbb5cf15787d1ab1398cc7

fix typo

view details

push time in 2 months

push eventeasynlp/easynlp

bentrevett

commit sha 9f28968b45e937f506631fb86e1fb4a4ba0d23d1

refactor cli from easynlpserver to easynlp

view details

Ben Trevett

commit sha 08c0f7202675636673bb506b66abe26cb7ddd2cd

Merge pull request #4 from easynlp/refactor_easynlpserver refactor cli from easynlpserver to easynlp

view details

push time in 2 months

PR opened easynlp/easynlp

refactor cli from easynlpserver to easynlp
+364 -364

0 comment

4 changed files

pr created time in 2 months

create barncheasynlp/easynlp

branch : refactor_easynlpserver

created branch time in 2 months

startedopenai/gym

started time in 2 months

startedd2l-ai/d2l-en

started time in 2 months

issue commentbentrevett/pytorch-seq2seq

Tutorial 6: Attention is all you need

Training large Transformer models still requires quite a few tricks to make them work properly.

The main two techniques I'd recommend looking into are initialization and learning rate schedulers.

A common way to initialize Transformers, which I'd recommend, is using a truncated normal distribution with a std of 0.02.

For learning rate schedulers, a lot of people use a "warm-up" stage, where the learning rate linearly increased from zero to some maximum value, followed by an annealing stage where the learning rate is then slowly decreased (usually linearly too).

Hannibal046

comment created time in 2 months

push eventkonfuzio-ai/document-ai-python-sdk

bentrevett

commit sha 308d61d827de098dd7ad50350a305ce9c8300f9d

fix template type hint

view details

push time in 2 months

push eventkonfuzio-ai/document-ai-python-sdk

bentrevett

commit sha 1792d1328d7d1cb6f9890bc9b958e6d6f6611382

fix circular import

view details

push time in 2 months