profile
viewpoint
brett koonce brettkoonce @quarkworks columbia, mo convolutionalneuralnetworkswithswift.com pobody's nerfect

fork silky/markserv

🏁 serve markdown as html (GitHub style), index directories, live-reload as you edit

https://f1lt3r.io

fork in 3 hours

PR closed tensorflow/swift-models

Reviewers
Wavenet model for audio generation

This implements a first cut Wavenet model for the audio generation task on the VCTK dataset. This uses the Python + Tensorflow version here as a reference implementation.

At the moment only the training loop works and there are some limitations / missing features that need to be addressed:

  • [ ] Add PaddingFIFOQueue to correctly produce batches from data files
  • [ ] Add generation step to generate new audio samples
  • [ ] Add preprocessing function to trim silent sections of audio
  • [ ] Make model configurable with command line args

Additionally, I have some ideas for future improvements / enhancements that I can include as follow up PRs:

  • Uselibrosa instead of the simplistic pydub library in Python for reading and processing audio. Initial attempts resulted in the following error:
Assertion failed: (PassInf && "Expected all immutable passes to be initialized"), function addImmutablePass, file /Users/buildbot/miniconda3/conda-bld/llvmdev_1556270736866/work/lib/IR/LegacyPassManager.cpp, line 849.
Abort trap: 6
  • Possibly use Swift-native audio libraries like AVFoundation? Initial attempts at this caused linker issues:
dyld: Symbol not found $<symbol-name>
expected in <path-to-swift-toolchain>/AVFoundation.swift
  • Add XLA support
  • Add support for more tasks and datasets
  • Add scalar input mode (non one-hot inputs)
  • Add L2 regularization for weights
  • Factor out some of the dataset and audio reading code into common utils

I'd love to get some early feedback on this!

+740 -0

7 comments

8 changed files

akshaan

pr closed time in 3 hours

pull request commenttensorflow/swift-models

Wavenet model for audio generation

Closing this out since it's outdated. I'll pull in updates from swift-apis etc. and reopen shortly!

akshaan

comment created time in 3 hours

push eventEleutherAI/gpt-neo

Leo Gao

commit sha ae71dddeb6c3ad102961dc7156bbc9943e393ecd

Update paper.md

view details

push time in 4 hours

fork silky/K3D-jupyter

K3D lets you create 3D plots backed by WebGL with high-level API (surfaces, isosurfaces, voxels, mesh, cloud points, vtk objects, volume renderer, colormaps, etc). The primary aim of K3D-jupyter is to be easy for use as stand alone package like matplotlib, but also to allow interoperation with existing libraries as VTK.

fork in 6 hours

fork silky/serialport

Cross platform haskell library for using the serial port

https://hackage.haskell.org/package/serialport

fork in 12 hours

startedDeepVoltaire/AutoAugment

started time in 13 hours

fork leogao2/scaling-experiments

Experiments related to scaling laws for language models.

fork in 14 hours

pull request commenttensorflow/swift-apis

Adding dilations argument inside DepthwiseConv2D API

@diwakar-vsingh - Right now, Colab uses Swift for TensorFlow toolchains with tensorflow/swift-apis as part of them. That means that you only get the swift-apis capabilities that correspond with those present at the time the toolchain was compiled (in this case, the 0.12 release). It's a difficult process to update the Colab images, so we only do that on stable releases.

If you instead develop locally, using our nightly toolchains, you'll get the latest APIs included in those. Or build swift-apis using CMake locally from a Swift.org toolchain.

diwakar-vsingh

comment created time in 16 hours

pull request commenttensorflow/swift-apis

Adding dilations argument inside DepthwiseConv2D API

@BradLarson I created the new colab notebook to test out the updated 'depthwiseConv2D()' function. However, I am not able to add a dilation argument inside it. When we import TensorFlow, don't we get the most up-to-date APIs?

diwakar-vsingh

comment created time in 19 hours

fork leogao2/BIG-bench

Beyond the Imitation Game collaborative benchmark for enormous language models

fork in 20 hours

push eventtensorflow/swift

Brad Larson

commit sha 3ef4fc7aaeeffa0420e2c357be95cc5ae531fccd

Add Python interoperability document. (#601)

view details

push time in 20 hours

PR merged tensorflow/swift

Add Python interoperability document to guides

This copies the existing Python interoperability document into our main list of guides, and inserts an entry for it in the table of contents. The interoperability document has been slightly edited to modernize the text, links, and some of the code examples to match the current state of the project.

+271 -0

0 comment

2 changed files

BradLarson

pr closed time in 20 hours

PR opened tensorflow/swift

Add Python interoperability document to guides

This copies the existing Python interoperability document into our main list of guides, and inserts an entry for it in the table of contents. The interoperability document has been slightly edited to modernize the text, links, and some of the code examples to match the current state of the project.

+271 -0

0 comment

2 changed files

pr created time in a day

startedritchieng/the-incredible-pytorch

started time in a day

pull request commenttensorflow/swift-apis

Adding dilations argument inside DepthwiseConv2D API

It's been my pleasure to contribute. I'll open up another PR in few weeks with DeepLab implementation.

diwakar-vsingh

comment created time in a day

push eventEleutherAI/gpt-neo

Leo Gao

commit sha fd5e02d4ac7a181a513ba431fcc83d9716fc9ab6

Update paper.bib

view details

push time in a day

push eventEleutherAI/gpt-neo

Leo Gao

commit sha e3f359fff45087eeb609ab3a5587e91a2ffacc21

Update paper.bib

view details

push time in a day

push eventtensorflow/swift-apis

Diwakar Singh

commit sha 02f31d531cbbfbd72de2b2f288f24f8645ee5bcb

Adding dilations argument inside DepthwiseConv2D API (#1129) * Update NN.swift Add dilations argument inside depthwiseConv2D function. * Update Convolutional.swift Updated depthwiseConv2D struct to include the dilations functionality as well. * Update Convolutional.swift Dilations argument added in SeparableConv2D * Updating X10 tests with dilations parameter. * Update ops_test.swift Correcting dilation argument inside depthwiseConv2D function. * Update NN.swift Update depthwiseconv2D() function's strides and dilations argument with default (1, 1, 1, 1) value. Co-authored-by: Brad Larson <bradlarson@google.com>

view details

push time in a day

PR merged tensorflow/swift-apis

Adding dilations argument inside DepthwiseConv2D API

Extended the depthwiseConv2D struct to control dilation rate as well.

+71 -34

18 comments

3 changed files

diwakar-vsingh

pr closed time in a day

push eventEleutherAI/gpt-neo

Leo Gao

commit sha 83080b69bb4acc78183db09ccef2e79b20ba8582

Create paper.bib

view details

push time in a day

create barnchEleutherAI/gpt-neo

branch : joss-paper

created branch time in a day

fork silky/talks

Miscellaneous talks container, select branch for particular talk

fork in a day

pull request commenttensorflow/swift-models

A higher-level interface to model checkpointing

I'm merging this in for now, to allow for the documentation to be accessible and to unblock work on improvements that depend on this.

BradLarson

comment created time in a day

push eventtensorflow/swift-models

Brad Larson

commit sha 339752a63b83a9fc4c8ce30ce67e834ea5735ab6

A higher-level interface to model checkpointing (#631) * Creating generalized tensor extraction for models, reworking GPT-2 to use this. * Adding unit tests for model checkpoint writing. * Added checkpoint reading to Checkpointable. * Minor access level adjustment. * Adding checkpointing overview documentation.

view details

push time in a day

PR merged tensorflow/swift-models

A higher-level interface to model checkpointing

CheckpointReader and CheckpointWriter provide the building blocks for reading and writing model tensors to TensorFlow v2 checkpoint files, but to use them you have had to write model-specific code for extracting Tensors and their names, as well as placing parameters back into the model. This introduces a higher-level API that makes it easy to checkpoint and restore simpler models, and provides customization options for more complex models or for matching the naming schemes of external frameworks.

Writing a checkpoint becomes as easy as

extension LeNet: Checkpointable {}

let model = LeNet()

try model.writeCheckpoint(to: directory, name: "LeNet")

and reading

var newModel = LeNet()

try newModel.readCheckpoint(from: directory, name: "LeNet")

This is handled via the new Checkpointable protocol, which builds on KeyPathIterable. If you extend a model to be Checkpointable you will gain the writeCheckpoint(to:...) and readCheckpoint(from:...) methods for writing and reading checkpoints. By default, this will generate path names for each Tensor<Float> within your model using a naming scheme where each level in the model is named by its property name and separated by "/". For example: conv1/filter.

This can be overridden in a number of ways. The internal separator string can be changed by providing a different checkpointSeparator in the protocol conformance. Specific tensors within the model can be ignored for checkpointing by providing them in ignoredTensorPaths in the protocol conformance.

For the greatest flexibility, customized String -> String mapping between internal tensor path names and those in the on-disk checkpoint can be provided as a default for the model via tensorNameMap in the protocol conformance, or as a parameter when reading or writing a specific checkpoint. This allows for interoperability between a wide variety of frameworks and model naming conventions.

Documentation and tests have been provided for checkpointing overall, and this new functionality in particular. This is now ready for review.

+525 -127

0 comment

10 changed files

BradLarson

pr closed time in a day

more