profile
viewpoint

keras-team/keras 48391

Deep Learning for humans

fchollet/deep-learning-with-python-notebooks 10489

Jupyter notebooks for the code samples of the book "Deep Learning with Python"

fchollet/deep-learning-models 6315

Keras code and weights files for popular deep learning models.

tensorflow/docs 3394

TensorFlow documentation

fchollet/keras-resources 3043

Directory of tutorials and open-source code repositories for working with Keras, the Python deep learning library

keras-team/keras-tuner 1910

Hyperparameter tuning for humans

keras-team/keras-contrib 1386

Keras community contributions

fchollet/ARC 1262

The Abstraction and Reasoning Corpus

keras-team/keras-docs-zh 610

Chinese (zh-cn) translation of the Keras documentation.

fchollet/hualos 357

Keras Total Visualization project

push eventkeras-team/keras-io

François Chollet

commit sha e0f87f14aaf4c27f264610d36fe828dc9046b060

Remove spurious warning

view details

push time in 15 hours

pull request commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

FYI, I had to make a number of small changes to the example, in particular I removed the TFP dependency. I will publish the page shortly. Thanks again!

ADMoreau

comment created time in 15 hours

push eventkeras-team/keras-io

Francois Chollet

commit sha 2b1e8bd00a6117339e55b6a2babe5467fb8795cf

add generated files for pixelcnn

view details

push time in 15 hours

push eventkeras-team/keras-io

François Chollet

commit sha 69edb66e6e23aa0bf1de27a66b9bb6b88031fd07

Remove TFP dependency in pixelcnn example

view details

push time in 15 hours

push eventkeras-team/keras-io

François Chollet

commit sha b53c38db2ff5cdc614a783b5f791c18492026681

Move imports in pixelcnn example

view details

François Chollet

commit sha 863582da5672901047a009c8eae6cdbade513d47

Merge branch 'master' of github.com:keras-team/keras-io

view details

push time in 15 hours

push eventkeras-team/keras-io

Francois Chollet

commit sha acd9e0f7db6db39266b68cacdb7c4e49066a0a43

remove tmp files incorrectly comitted

view details

push time in 16 hours

push eventkeras-team/keras-io

François Chollet

commit sha a97eace61496a20d041b2f1bb07704e67baaf62a

Small update to PixelCNN example

view details

push time in 16 hours

push eventkeras-team/keras-io

ADMoreau

commit sha 3c43b64292f30edf7c353755223cb790531714e8

added PixelCNN example, added pathlib to requirements (#45) * added PixelCNN example, added pathlib to requirements * reformatted * attempt to compile to website * added example * example done * necessary changes * fixed formatting * added cleaned example * cleanup * Style nits * issue resolution * issue resolution * added cleaned example * full functionality * remerged nits * fixed formatting * fixed formatting * blackened file * Style nits * removed redundant relu * added cleaned example * cleanup * added cleaned example * cleanup Co-authored-by: François Chollet <francois.chollet@gmail.com>

view details

push time in 16 hours

pull request commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

ImportError: cannot import name 'naming'

This type of error occurs when you have cached files remaining from a conflicting installation. Uninstalling everything (or just create a new virtual env) would fix it.

ADMoreau

comment created time in 16 hours

push eventkeras-team/keras-io

François Chollet

commit sha 367ee2868d5b6953c28c8acb60ed3563826c252c

Update guides files

view details

push time in 17 hours

push eventkeras-team/keras-io

SamMinkowicz

commit sha c90aaeadc96074142d55039c7864dc98427e7fdd

Fixed broken links to keras api (#52) The links were broken for keras documentation of LSTM, GRU, and bidirectional recurrent layers.

view details

push time in 17 hours

PR merged keras-team/keras-io

Fixed broken links to keras api in Working with RNNs developer guide

The links to keras documentation of LSTM, GRU, and bidirectional recurrent layers were broken.

+3 -3

0 comment

1 changed file

SamMinkowicz

pr closed time in 17 hours

pull request commentkeras-team/keras-io

PointNet example

Also, about the CI check: please run the black code formatter on the file.

dgriffiths3

comment created time in 17 hours

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/26+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+# Point cloud classification++## Introduction++Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup++Install trimesh for mesh processing and sampling+If using colab run `!pip install trimesh`+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from matplotlib import pyplot as plt++"""+## Load dataset++We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)++fig = plt.figure(figsize=(5, 5))+ax = fig.add_subplot(111, projection="3d")+ax.scatter(points[:, 0], points[:, 1], points[:, 2])+ax.set_axis_off()+plt.show()++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label and use a dictionary to recall this later.+"""++def parse_dataset(num_points=2048):++    train_points = []+    train_labels = []+    test_points = []+    test_labels = []+    class_map = {}+    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):+        print("processing: %s" % folder)+        # store folder name with ID so we can retrieve later+        class_map[i] = folder.split("/")[-1]+        # gather all files+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_points.append(trimesh.load(f).sample(num_points))+            train_labels.append(i)++        for f in test_files:+            test_points.append(trimesh.load(f).sample(num_points))+            test_labels.append(i)++    return (+        np.array(train_points),+        np.array(test_points),+        np.array(train_labels),+        np.array(test_labels),+        class_map,+    )++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++NUM_POINTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++train_points, test_points, train_labels, test_labels, CLASS_MAP = parse_dataset(+    NUM_POINTS+)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter and shuffle the train dataset.+"""++def augment(points, label):+    # jitter points+    points += tf.random.uniform(points.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    points = tf.random.shuffle(points)+    return points, label++train_dataset = tf.data.Dataset.from_tensor_slices(+    (train_points, train_labels))+test_dataset = tf.data.Dataset.from_tensor_slices((test_points, test_labels))++train_dataset = train_dataset.shuffle(+    len(train_points)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(test_points)).batch(BATCH_SIZE)++"""+### Build a model++Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation.+"""++def conv_bn(x, filters):+    x = layers.Conv1D(filters, kernel_size=1, padding="valid")(x)+    x = layers.BatchNormalization(momentum=0.0)(x)+    return layers.Activation("relu")(x)++def dense_bn(x, filters):+    x = layers.Dense(filters)(x)+    x = layers.BatchNormalization(momentum=0.0)(x)+    return layers.Activation("relu")(x)++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (T-net). The T-net aims to learn an affine transformation matrix by its own mini+network. The T-net is used twice. The first time to transform the input features (n, 3)+into a canonical representation. The second is an affine transformation for alignment in+feature space (n, 3). As per the original paper we constrain the transformation to be+close to an orthogonal matrix (i.e. ||X*X^T - I|| = 0).+"""++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = tf.eye(self.k)++    def __call__(self, x):+        x = tf.reshape(x, (-1, self.k, self.k))+        xxt = tf.tensordot(x, x, axes=(2, 2))+        xxt = tf.reshape(xxt, (-1, self.k, self.k))+        return tf.reduce_sum(self.l2reg * tf.square(xxt - self.eye))++"""+ We can define the T-net as a functional model.+"""++def tnet(num_points, k, name):

num_features sounds good to me, clear & descriptive.

dgriffiths3

comment created time in 18 hours

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/26+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+# Point cloud classification++## Introduction++Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup++Install trimesh for mesh processing and sampling+If using colab run `!pip install trimesh`+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from matplotlib import pyplot as plt++"""+## Load dataset++We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)++fig = plt.figure(figsize=(5, 5))+ax = fig.add_subplot(111, projection="3d")+ax.scatter(points[:, 0], points[:, 1], points[:, 2])+ax.set_axis_off()+plt.show()++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label and use a dictionary to recall this later.+"""++def parse_dataset(num_points=2048):++    train_points = []+    train_labels = []+    test_points = []+    test_labels = []+    class_map = {}+    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):+        print("processing: %s" % folder)+        # store folder name with ID so we can retrieve later+        class_map[i] = folder.split("/")[-1]+        # gather all files+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_points.append(trimesh.load(f).sample(num_points))+            train_labels.append(i)++        for f in test_files:+            test_points.append(trimesh.load(f).sample(num_points))+            test_labels.append(i)++    return (+        np.array(train_points),+        np.array(test_points),+        np.array(train_labels),+        np.array(test_labels),+        class_map,+    )++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++NUM_POINTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++train_points, test_points, train_labels, test_labels, CLASS_MAP = parse_dataset(+    NUM_POINTS+)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter and shuffle the train dataset.+"""++def augment(points, label):+    # jitter points+    points += tf.random.uniform(points.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    points = tf.random.shuffle(points)+    return points, label++train_dataset = tf.data.Dataset.from_tensor_slices(+    (train_points, train_labels))+test_dataset = tf.data.Dataset.from_tensor_slices((test_points, test_labels))++train_dataset = train_dataset.shuffle(+    len(train_points)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(test_points)).batch(BATCH_SIZE)++"""+### Build a model++Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation.+"""++def conv_bn(x, filters):+    x = layers.Conv1D(filters, kernel_size=1, padding="valid")(x)+    x = layers.BatchNormalization(momentum=0.0)(x)+    return layers.Activation("relu")(x)++def dense_bn(x, filters):+    x = layers.Dense(filters)(x)+    x = layers.BatchNormalization(momentum=0.0)(x)+    return layers.Activation("relu")(x)++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (T-net). The T-net aims to learn an affine transformation matrix by its own mini+network. The T-net is used twice. The first time to transform the input features (n, 3)+into a canonical representation. The second is an affine transformation for alignment in+feature space (n, 3). As per the original paper we constrain the transformation to be+close to an orthogonal matrix (i.e. ||X*X^T - I|| = 0).+"""++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = tf.eye(self.k)++    def __call__(self, x):+        x = tf.reshape(x, (-1, self.k, self.k))+        xxt = tf.tensordot(x, x, axes=(2, 2))+        xxt = tf.reshape(xxt, (-1, self.k, self.k))+        return tf.reduce_sum(self.l2reg * tf.square(xxt - self.eye))++"""+ We can define the T-net as a functional model.+"""++def tnet(num_points, k, name):

Please use a more descriptive name for k.

Going back to the comment about using a functional approach -- return a Model instance is fine, but in this case it turns out you don't need to do it, because you never use that instance as a model (e.g. fit()). So here converting this function to just apply layers to an input (like conv_bn) would be beneficial, since it would mean you don't need to pass num_points.

dgriffiths3

comment created time in 18 hours

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):

e.g. what is k?

dgriffiths3

comment created time in 18 hours

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.

A function that returns a Model instance is a perfectly fine pattern, no problem there.

dgriffiths3

comment created time in 18 hours

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: [Siddhartha Banerjee](https://twitter.com/sidd2006)+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using a model trained on Movielens dataset.+"""+"""+## Introduction++This example demonstrates+[Collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering)+using the [Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+The MovieLens ratings dataset lists the ratings given by a set of users to a set of movies.+Our goal is to be able to predict ratings for movies an user has not yet watched.+The movies with the highest predicted ratings can then be recommended to the user.++The steps in the model are as follows:++1. Map user ID to a "user vector" via an embedding matrix+2. Map movie ID to a "movie vector" via an embedding matrix+3. Compute the dot product between the user vector and movie vector, to obtain+the a match score between the user and the movie (predicted rating).+4. Train the embeddings via gradient descent using all known user-movie pairs.++**References:**++- [Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/371920.372071)+- [Neural Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/3038912.3052569)+"""++import pandas as pd+import numpy as np+from zipfile import ZipFile+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from pathlib import Path+import matplotlib.pyplot as plt++"""+## First, load the data and apply preprocessing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# Use the ratings.csv file+movielens_data_file_url = (+    "http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+)+movielens_zipped_file = keras.utils.get_file(+    "ml-latest-small.zip", movielens_data_file_url, extract=False+)+keras_datasets_path = Path(movielens_zipped_file).parents[0]+movielens_dir = keras_datasets_path / "ml-latest-small"++# Only extract the data the first time the script is run.+if not movielens_dir.exists():+    with ZipFile(movielens_zipped_file, "r") as zip:+        # Extract files+        print("Extracting all the files now...")+        zip.extractall(path=keras_datasets_path)+        print("Done!")++ratings_file = movielens_dir / "ratings.csv"+df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies as integer indices.+"""+user_ids = df["userId"].unique().tolist()+user2user_encoded = {x: i for i, x in enumerate(user_ids)}+userencoded2user = {i: x for i, x in enumerate(user_ids)}+movie_ids = df["movieId"].unique().tolist()+movie2movie_encoded = {x: i for i, x in enumerate(movie_ids)}+movie_encoded2movie = {i: x for i, x in enumerate(movie_ids)}+df["user"] = df["userId"].map(user2user_encoded)+df["movie"] = df["movieId"].map(movie2movie_encoded)++num_users = len(user2user_encoded)+num_movies = len(movie_encoded2movie)+df["rating"] = df["rating"].values.astype(np.float32)+# min and max ratings will be used to normalize the ratings later+min_rating = min(df["rating"])+max_rating = max(df["rating"])++print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        num_users, num_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""+df = df.sample(frac=1, random_state=42)+x = df[["user", "movie"]].values+# Normalize the targets between 0 and 1. Makes it easy to train.+y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values+# Assuming training on 90% of the data and validating on 10%.+train_indices = int(0.9 * df.shape[0])+x_train, x_test, y_train, y_test = (+    x[:train_indices],+    x[train_indices:],+    y[:train_indices],+    y[train_indices:],+)++"""+## Create the model++We embed both users and movies in to 50-dimensional vectors.++The model computes a match score between user and movie embeddings via a dot product,+and adds a per-movie and per-user bias. The match score is scaled to the `[0, 1]`+interval via a sigmoid (since our ratings are normalized to this range).+"""+# Setting a random seed for reproducibility+tf.random.set_seed(1234)++EMBEDDING_SIZE = 50+++class RecommenderNet(keras.Model):+    def __init__(self, num_users, num_movies, embedding_size, **kwargs):+        super(RecommenderNet, self).__init__(**kwargs)+        self.num_users = num_users+        self.num_movies = num_movies+        self.embedding_size = embedding_size+        self.user_embedding = layers.Embedding(+            num_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.user_bias = layers.Embedding(num_users, 1)+        self.movie_embedding = layers.Embedding(+            num_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.movie_bias = layers.Embedding(num_movies, 1)++    def call(self, inputs):+        user_vector = self.user_embedding(inputs[:, 0])+        user_bias = self.user_bias(inputs[:, 0])+        movie_vector = self.movie_embedding(inputs[:, 1])+        movie_bias = self.movie_bias(inputs[:, 1])+        dot_user_movie = tf.tensordot(user_vector, movie_vector, 2)+        # Add all the components (including bias)+        x = dot_user_movie + user_bias + movie_bias+        # The sigmoid activation forces the rating to between 0 and 1+        return tf.nn.sigmoid(x)+++inputs = layers.Input(shape=(2,))+model = RecommenderNet(num_users, num_movies, EMBEDDING_SIZE)+outputs = model(inputs)+opt = keras.optimizers.Adam(lr=0.001)+model.compile(loss=tf.keras.losses.BinaryCrossentropy(), optimizer=opt)++"""+## Train the model based on the data split+"""+history = model.fit(+    x=x_train,+    y=y_train,+    batch_size=64,+    epochs=5,+    verbose=1,+    validation_data=(x_test, y_test),+)++"""+## Plot training and test loss+"""+plt.plot(history.history["loss"])+plt.plot(history.history["val_loss"])+plt.title("model loss")+plt.ylabel("loss")+plt.xlabel("epoch")+plt.legend(["train", "test"], loc="upper left")+plt.show()++"""+## Show top 10 movie recommendations to an user+"""++movie_df = pd.read_csv(movielens_dir / "movies.csv")++# Let us get an user and see the top recommendations.+user_id = df.userId.sample(1).iloc[0]+movies_watched_by_user = df[df.userId == user_id].movieId.values+movies_not_watched = movie_df[~movie_df["movieId"].isin(movies_watched_by_user)][+    "movieId"+]+movies_not_watched = list(+    set(movies_not_watched).intersection(set(movie2movie_encoded.keys()))+)+movies_not_watched = [[movie2movie_encoded.get(x)] for x in movies_not_watched]+user_encoder = user2user_encoded.get(user_id)+user_movie_array = np.hstack(+    ([[user_id]] * len(movies_not_watched), movies_not_watched)+)+ratings = model.predict(user_movie_array).flatten()+top_ratings_indices = ratings.argsort()[-10:][::-1]+recommended_movie_ids = [movie_encoded2movie.get(x) for x in top_ratings_indices]++print("Here are the top 10 movie recommendations for user: {}".format(user_id))

Please also print the titles of the movies were rated highly by the user, so the readers can see the relationship between liked movies and the new recommendations

siddBanPsu

comment created time in 18 hours

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: [Siddhartha Banerjee](https://twitter.com/sidd2006)+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using a model trained on Movielens dataset.+"""+"""+## Introduction++This example demonstrates+[Collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering)+using the [Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+The MovieLens ratings dataset lists the ratings given by a set of users to a set of movies.+Our goal is to be able to predict ratings for movies an user has not yet watched.+The movies with the highest predicted ratings can then be recommended to the user.++The steps in the model are as follows:++1. Map user ID to a "user vector" via an embedding matrix+2. Map movie ID to a "movie vector" via an embedding matrix+3. Compute the dot product between the user vector and movie vector, to obtain+the a match score between the user and the movie (predicted rating).+4. Train the embeddings via gradient descent using all known user-movie pairs.++**References:**++- [Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/371920.372071)+- [Neural Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/3038912.3052569)+"""++import pandas as pd+import numpy as np+from zipfile import ZipFile+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from pathlib import Path+import matplotlib.pyplot as plt++"""+## First, load the data and apply preprocessing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# Use the ratings.csv file+movielens_data_file_url = (+    "http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+)+movielens_zipped_file = keras.utils.get_file(+    "ml-latest-small.zip", movielens_data_file_url, extract=False+)+keras_datasets_path = Path(movielens_zipped_file).parents[0]+movielens_dir = keras_datasets_path / "ml-latest-small"++# Only extract the data the first time the script is run.+if not movielens_dir.exists():+    with ZipFile(movielens_zipped_file, "r") as zip:+        # Extract files+        print("Extracting all the files now...")+        zip.extractall(path=keras_datasets_path)+        print("Done!")++ratings_file = movielens_dir / "ratings.csv"+df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies as integer indices.+"""+user_ids = df["userId"].unique().tolist()+user2user_encoded = {x: i for i, x in enumerate(user_ids)}+userencoded2user = {i: x for i, x in enumerate(user_ids)}+movie_ids = df["movieId"].unique().tolist()+movie2movie_encoded = {x: i for i, x in enumerate(movie_ids)}+movie_encoded2movie = {i: x for i, x in enumerate(movie_ids)}+df["user"] = df["userId"].map(user2user_encoded)+df["movie"] = df["movieId"].map(movie2movie_encoded)++num_users = len(user2user_encoded)+num_movies = len(movie_encoded2movie)+df["rating"] = df["rating"].values.astype(np.float32)+# min and max ratings will be used to normalize the ratings later+min_rating = min(df["rating"])+max_rating = max(df["rating"])++print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        num_users, num_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""+df = df.sample(frac=1, random_state=42)+x = df[["user", "movie"]].values+# Normalize the targets between 0 and 1. Makes it easy to train.+y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values+# Assuming training on 90% of the data and validating on 10%.+train_indices = int(0.9 * df.shape[0])+x_train, x_test, y_train, y_test = (+    x[:train_indices],+    x[train_indices:],+    y[:train_indices],+    y[train_indices:],+)++"""+## Create the model++We embed both users and movies in to 50-dimensional vectors.++The model computes a match score between user and movie embeddings via a dot product,+and adds a per-movie and per-user bias. The match score is scaled to the `[0, 1]`+interval via a sigmoid (since our ratings are normalized to this range).+"""+# Setting a random seed for reproducibility+tf.random.set_seed(1234)++EMBEDDING_SIZE = 50+++class RecommenderNet(keras.Model):+    def __init__(self, num_users, num_movies, embedding_size, **kwargs):+        super(RecommenderNet, self).__init__(**kwargs)+        self.num_users = num_users+        self.num_movies = num_movies+        self.embedding_size = embedding_size+        self.user_embedding = layers.Embedding(+            num_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.user_bias = layers.Embedding(num_users, 1)+        self.movie_embedding = layers.Embedding(+            num_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.movie_bias = layers.Embedding(num_movies, 1)++    def call(self, inputs):+        user_vector = self.user_embedding(inputs[:, 0])+        user_bias = self.user_bias(inputs[:, 0])+        movie_vector = self.movie_embedding(inputs[:, 1])+        movie_bias = self.movie_bias(inputs[:, 1])+        dot_user_movie = tf.tensordot(user_vector, movie_vector, 2)+        # Add all the components (including bias)+        x = dot_user_movie + user_bias + movie_bias+        # The sigmoid activation forces the rating to between 0 and 1+        return tf.nn.sigmoid(x)+++inputs = layers.Input(shape=(2,))+model = RecommenderNet(num_users, num_movies, EMBEDDING_SIZE)+outputs = model(inputs)

You don't need to create an input. You can just do:

model = RecommenderNet(num_users, num_movies, EMBEDDING_SIZE)
model.compile(...)
model.fit(...)
siddBanPsu

comment created time in 18 hours

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: [Siddhartha Banerjee](https://twitter.com/sidd2006)+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using a model trained on Movielens dataset.+"""+"""+## Introduction++This example demonstrates+[Collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering)+using the [Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+The MovieLens ratings dataset lists the ratings given by a set of users to a set of movies.+Our goal is to be able to predict ratings for movies an user has not yet watched.+The movies with the highest predicted ratings can then be recommended to the user.++The steps in the model are as follows:++1. Map user ID to a "user vector" via an embedding matrix+2. Map movie ID to a "movie vector" via an embedding matrix+3. Compute the dot product between the user vector and movie vector, to obtain+the a match score between the user and the movie (predicted rating).+4. Train the embeddings via gradient descent using all known user-movie pairs.++**References:**++- [Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/371920.372071)+- [Neural Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/3038912.3052569)+"""++import pandas as pd+import numpy as np+from zipfile import ZipFile+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from pathlib import Path+import matplotlib.pyplot as plt++"""+## First, load the data and apply preprocessing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# Use the ratings.csv file+movielens_data_file_url = (+    "http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+)+movielens_zipped_file = keras.utils.get_file(+    "ml-latest-small.zip", movielens_data_file_url, extract=False+)+keras_datasets_path = Path(movielens_zipped_file).parents[0]+movielens_dir = keras_datasets_path / "ml-latest-small"++# Only extract the data the first time the script is run.+if not movielens_dir.exists():+    with ZipFile(movielens_zipped_file, "r") as zip:+        # Extract files+        print("Extracting all the files now...")+        zip.extractall(path=keras_datasets_path)+        print("Done!")++ratings_file = movielens_dir / "ratings.csv"+df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies as integer indices.+"""+user_ids = df["userId"].unique().tolist()+user2user_encoded = {x: i for i, x in enumerate(user_ids)}+userencoded2user = {i: x for i, x in enumerate(user_ids)}+movie_ids = df["movieId"].unique().tolist()+movie2movie_encoded = {x: i for i, x in enumerate(movie_ids)}+movie_encoded2movie = {i: x for i, x in enumerate(movie_ids)}+df["user"] = df["userId"].map(user2user_encoded)+df["movie"] = df["movieId"].map(movie2movie_encoded)++num_users = len(user2user_encoded)+num_movies = len(movie_encoded2movie)+df["rating"] = df["rating"].values.astype(np.float32)+# min and max ratings will be used to normalize the ratings later+min_rating = min(df["rating"])+max_rating = max(df["rating"])++print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        num_users, num_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""+df = df.sample(frac=1, random_state=42)+x = df[["user", "movie"]].values+# Normalize the targets between 0 and 1. Makes it easy to train.+y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values+# Assuming training on 90% of the data and validating on 10%.+train_indices = int(0.9 * df.shape[0])+x_train, x_test, y_train, y_test = (+    x[:train_indices],+    x[train_indices:],+    y[:train_indices],+    y[train_indices:],+)++"""+## Create the model++We embed both users and movies in to 50-dimensional vectors.++The model computes a match score between user and movie embeddings via a dot product,+and adds a per-movie and per-user bias. The match score is scaled to the `[0, 1]`+interval via a sigmoid (since our ratings are normalized to this range).+"""+# Setting a random seed for reproducibility+tf.random.set_seed(1234)

This is not meaningful; for reproducibility you to set both the global seed and per-op seeds. Since you don't do the latter, this does nothing. You can remove these 2 lines

siddBanPsu

comment created time in 18 hours

pull request commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

Are you observing any performance difference between tensordot and Dot?

siddBanPsu

comment created time in 18 hours

pull request commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

We can remove the warning logs by hand afterwards, so let's not worry too much about it. Is everything ready otherwise?

ADMoreau

comment created time in 19 hours

Pull request review commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

++**Author:** [ADMoreau](https://github.com/ADMoreau)<br>+**Date created:** 2020/05/17<br>+**Last modified:** 2020/05/23<br>+**Description:** PixelCNN implemented in Keras.+++<img class="k-inline-icon" src="https://colab.research.google.com/img/colab_favicon.ico"/> [**View in Colab**](https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/generative/ipynb/pixelcnn.ipynb)  <span class="k-dot">•</span><img class="k-inline-icon" src="https://github.com/favicon.ico"/> [**GitHub source**](https://github.com/keras-team/keras-io/blob/master/examples/generative/pixelcnn.py)++---+## Introduction++PixelCNN is a generative model proposed in 2016 by van den Oord et al.+(reference: [https://arxiv.org/abs/1606.05328](https://arxiv.org/abs/1606.05328)).+It is designed to generate images (or other data types) iteratively,+from an input vector where the probability distribution of prior elements dictates the+probability distribution of later elements. In the following example, images are generated+in this fashion, pixel-by-pixel, via a masked convolution kernel that only looks at data+from previously generated pixels (origin at the top left) to generate later pixels.+During inference, the output of the network is used as a probability ditribution+from which new pixel values are sampled to generate a new image+(here, with MNIST, the pixels values are either black or white).++++```python+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers++```++---+## Getting the Data++++```python+# Model / data parameters+num_classes = 10+input_shape = (28, 28, 1)+n_residual_blocks = 5+# The data, split between train and test sets+(x, _), (y, _) = keras.datasets.mnist.load_data()+# Concatenate all of the images together+data = np.concatenate((x, y), axis=0)+# Round all pixel values less than 33% of the max 256 value to 0+# anything above this value gets rounded up to 1 so that all values are either+# 0 or 1+data = np.where(data < (0.33 * 256), 0, 1)+data = data.astype(np.float32)++```++---+## Create two classes for the requisite Layers for the model++++```python+# The first layer is the PixelCNN layer. This layer simply+# builds on the 2D convolutional layer, but includes masking.+class PixelConvLayer(layers.Layer):+    def __init__(self, mask_type, **kwargs):+        super(PixelConvLayer, self).__init__()+        self.mask_type = mask_type+        self.conv = layers.Conv2D(**kwargs)++    def build(self, input_shape):+        # Build the conv2d layer to initialize kernel variables+        self.conv.build(input_shape)+        # Use the initialized kernel to create the mask+        kernel_shape = self.conv.kernel.get_shape()+        self.mask = np.zeros(shape=kernel_shape)+        self.mask[: kernel_shape[0] // 2, ...] = 1.0+        self.mask[kernel_shape[0] // 2, : kernel_shape[1] // 2, ...] = 1.0+        if self.mask_type == "B":+            self.mask[kernel_shape[0] // 2, kernel_shape[1] // 2, ...] = 1.0++    def call(self, inputs):+        self.conv.kernel.assign(self.conv.kernel * self.mask)+        return self.conv(inputs)+++# Next, we build our residual block layer.+# This is just a normal residual block, but base don the PixelConvLayer.+class ResidualBlock(keras.layers.Layer):+    def __init__(self, filters, **kwargs):+        super(ResidualBlock, self).__init__(**kwargs)+        self.activation = keras.layers.ReLU()+        self.conv1 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )+        self.pixel_conv = PixelConvLayer(+            mask_type="B",+            filters=filters // 2,+            kernel_size=3,+            activation="relu",+            padding="same",+        )+        self.conv2 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )++    def call(self, inputs):+        x = self.activation(inputs)+        x = self.conv1(x)+        x = self.pixel_conv(x)+        x = self.conv2(x)+        return keras.layers.add([inputs, x])+++```++---+## Build the model based on the original paper++++```python+inputs = keras.Input(shape=input_shape)+x = PixelConvLayer(mask_type="A", filters=128, kernel_size=7, padding="same")(inputs)++for _ in range(n_residual_blocks):+    x = ResidualBlock(filters=128)(x)++for _ in range(2):+    x = PixelConvLayer(+        mask_type="B",+        filters=128,+        kernel_size=1,+        strides=1,+        activation="relu",+        padding="valid",+    )(x)++out = keras.layers.Conv2D(+    filters=1, kernel_size=1, strides=1, activation="sigmoid", padding="valid"+)(x)++pixel_cnn = keras.Model(inputs, out)+adam = keras.optimizers.Adam(learning_rate=0.0001)+pixel_cnn.compile(optimizer=adam, loss="binary_crossentropy")++pixel_cnn.summary()+pixel_cnn.fit(x=data, y=data, batch_size=64, epochs=50, validation_split=0.1)++```++<div class="k-default-codeblock">+```+Model: "model"+_________________________________________________________________+Layer (type)                 Output Shape              Param #   +=================================================================+input_1 (InputLayer)         [(None, 28, 28, 1)]       0         +_________________________________________________________________+pixel_conv_layer (PixelConvL (None, 28, 28, 128)       6400      +_________________________________________________________________+residual_block (ResidualBloc (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_1 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_2 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_3 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_4 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+pixel_conv_layer_6 (PixelCon (None, 28, 28, 128)       16512     +_________________________________________________________________+pixel_conv_layer_7 (PixelCon (None, 28, 28, 128)       16512     +_________________________________________________________________+conv2d_18 (Conv2D)           (None, 28, 28, 1)         129       +=================================================================+Total params: 532,673+Trainable params: 532,673+Non-trainable params: 0+_________________________________________________________________+Epoch 1/50+  2/985 [..............................] - ETA: 1:02 - loss: 0.6944WARNING:tensorflow:Callbacks method `on_train_batch_end` is slow compared to the batch time. Check your callbacks.

Try pip install tf-nightly --upgrade, it seems your nightly build is old. We removed this warning a couple of weeks ago.

ADMoreau

comment created time in 19 hours

Pull request review commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

++**Author:** [ADMoreau](https://github.com/ADMoreau)<br>+**Date created:** 2020/05/17<br>+**Last modified:** 2020/05/23<br>+**Description:** PixelCNN implemented in Keras.+++<img class="k-inline-icon" src="https://colab.research.google.com/img/colab_favicon.ico"/> [**View in Colab**](https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/generative/ipynb/pixelcnn.ipynb)  <span class="k-dot">•</span><img class="k-inline-icon" src="https://github.com/favicon.ico"/> [**GitHub source**](https://github.com/keras-team/keras-io/blob/master/examples/generative/pixelcnn.py)++---+## Introduction++PixelCNN is a generative model proposed in 2016 by van den Oord et al.+(reference: [https://arxiv.org/abs/1606.05328](https://arxiv.org/abs/1606.05328)).+It is designed to generate images (or other data types) iteratively,+from an input vector where the probability distribution of prior elements dictates the+probability distribution of later elements. In the following example, images are generated+in this fashion, pixel-by-pixel, via a masked convolution kernel that only looks at data+from previously generated pixels (origin at the top left) to generate later pixels.+During inference, the output of the network is used as a probability ditribution+from which new pixel values are sampled to generate a new image+(here, with MNIST, the pixels values are either black or white).++++```python+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers++```++---+## Getting the Data++++```python+# Model / data parameters+num_classes = 10+input_shape = (28, 28, 1)+n_residual_blocks = 5+# The data, split between train and test sets+(x, _), (y, _) = keras.datasets.mnist.load_data()+# Concatenate all of the images together+data = np.concatenate((x, y), axis=0)+# Round all pixel values less than 33% of the max 256 value to 0+# anything above this value gets rounded up to 1 so that all values are either+# 0 or 1+data = np.where(data < (0.33 * 256), 0, 1)+data = data.astype(np.float32)++```++---+## Create two classes for the requisite Layers for the model++++```python+# The first layer is the PixelCNN layer. This layer simply+# builds on the 2D convolutional layer, but includes masking.+class PixelConvLayer(layers.Layer):+    def __init__(self, mask_type, **kwargs):+        super(PixelConvLayer, self).__init__()+        self.mask_type = mask_type+        self.conv = layers.Conv2D(**kwargs)++    def build(self, input_shape):+        # Build the conv2d layer to initialize kernel variables+        self.conv.build(input_shape)+        # Use the initialized kernel to create the mask+        kernel_shape = self.conv.kernel.get_shape()+        self.mask = np.zeros(shape=kernel_shape)+        self.mask[: kernel_shape[0] // 2, ...] = 1.0+        self.mask[kernel_shape[0] // 2, : kernel_shape[1] // 2, ...] = 1.0+        if self.mask_type == "B":+            self.mask[kernel_shape[0] // 2, kernel_shape[1] // 2, ...] = 1.0++    def call(self, inputs):+        self.conv.kernel.assign(self.conv.kernel * self.mask)+        return self.conv(inputs)+++# Next, we build our residual block layer.+# This is just a normal residual block, but base don the PixelConvLayer.+class ResidualBlock(keras.layers.Layer):+    def __init__(self, filters, **kwargs):+        super(ResidualBlock, self).__init__(**kwargs)+        self.activation = keras.layers.ReLU()+        self.conv1 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )+        self.pixel_conv = PixelConvLayer(+            mask_type="B",+            filters=filters // 2,+            kernel_size=3,+            activation="relu",+            padding="same",+        )+        self.conv2 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )++    def call(self, inputs):+        x = self.activation(inputs)+        x = self.conv1(x)+        x = self.pixel_conv(x)+        x = self.conv2(x)+        return keras.layers.add([inputs, x])+++```++---+## Build the model based on the original paper++++```python+inputs = keras.Input(shape=input_shape)+x = PixelConvLayer(mask_type="A", filters=128, kernel_size=7, padding="same")(inputs)++for _ in range(n_residual_blocks):+    x = ResidualBlock(filters=128)(x)++for _ in range(2):+    x = PixelConvLayer(+        mask_type="B",+        filters=128,+        kernel_size=1,+        strides=1,+        activation="relu",+        padding="valid",+    )(x)++out = keras.layers.Conv2D(+    filters=1, kernel_size=1, strides=1, activation="sigmoid", padding="valid"+)(x)++pixel_cnn = keras.Model(inputs, out)+adam = keras.optimizers.Adam(learning_rate=0.0001)+pixel_cnn.compile(optimizer=adam, loss="binary_crossentropy")++pixel_cnn.summary()+pixel_cnn.fit(x=data, y=data, batch_size=64, epochs=50, validation_split=0.1)++```++<div class="k-default-codeblock">+```+Model: "model"+_________________________________________________________________+Layer (type)                 Output Shape              Param #   +=================================================================+input_1 (InputLayer)         [(None, 28, 28, 1)]       0         +_________________________________________________________________+pixel_conv_layer (PixelConvL (None, 28, 28, 128)       6400      +_________________________________________________________________+residual_block (ResidualBloc (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_1 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_2 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_3 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_4 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+pixel_conv_layer_6 (PixelCon (None, 28, 28, 128)       16512     +_________________________________________________________________+pixel_conv_layer_7 (PixelCon (None, 28, 28, 128)       16512     +_________________________________________________________________+conv2d_18 (Conv2D)           (None, 28, 28, 1)         129       +=================================================================+Total params: 532,673+Trainable params: 532,673+Non-trainable params: 0+_________________________________________________________________+Epoch 1/50+  2/985 [..............................] - ETA: 1:02 - loss: 0.6944WARNING:tensorflow:Callbacks method `on_train_batch_end` is slow compared to the batch time. Check your callbacks.

Just pip install tf-nightly

ADMoreau

comment created time in a day

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: [Siddhartha Banerjee](https://twitter.com/sidd2006)+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using embedding-based model on Movielens dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering)+using [Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+The MovieLens ratings dataset includes users and movies with the corresponding ratings.+The goal is to be able to predict ratings for movies an user has not yet watched.+Thereafter, the movies with higher predicted ratings will be recommended to the user.++The steps in the model are as follows:++1. Feed user identifier and movie identifier to the model.+2. Use an embedding matrix to obtain the corresponding user and movie embeddings.+3. Obtain a dot product of the embeddings, the get a score and add bias.+4. Use a sigmoid activation to scale the ratings between 0-1 scale.+5. Convert the scaled output to a number between minimum and maximum rating.++**References:**++- [Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/371920.372071)+- [Neural Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/3038912.3052569)+"""++import pandas as pd+import numpy as np+from zipfile import ZipFile+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from pathlib import Path+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file+movielens_data_file_url = (+    "http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+)+movielens_zipped_file = keras.utils.get_file(+    "ml-latest-small.zip", movielens_data_file_url, extract=False+)+keras_datasets_path = Path(movielens_zipped_file).parents[0]+movielens_dir = keras_datasets_path / "ml-latest-small"++# Run the following piece only if it is the first time.+# Otherwise, data should already be there.+# In case the MovieLens 100k data has changed over time,+# then please delete the folder in datasets manually and rerun.+if not movielens_dir.exists():+    with ZipFile(movielens_zipped_file, "r") as zip:+        # extracting all the files+        print("Extracting all the files now...")+        zip.extractall(path=keras_datasets_path)+        print("Done!")++ratings_file = movielens_dir / "ratings.csv"+df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""+user_ids = df["userId"].unique().tolist()+user2user_encoded = {x: i for i, x in enumerate(user_ids)}+userencoded2user = {i: x for i, x in enumerate(user_ids)}+movie_ids = df["movieId"].unique().tolist()+movie2movie_encoded = {x: i for i, x in enumerate(movie_ids)}+movie_encoded2movie = {i: x for i, x in enumerate(movie_ids)}+df["user"] = df["userId"].map(user2user_encoded)+df["movie"] = df["movieId"].map(movie2movie_encoded)++num_users = len(user2user_encoded)+num_movies = len(movie_encoded2movie)+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        num_users, num_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""+df = df.sample(frac=1, random_state=42)+x = df[["user", "movie"]].values+# Normalize the targets between 0 and 1. Makes it easy to train.+y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values+# Assuming training on 90% of the data and validating on 10%.+train_indices = int(0.9 * df.shape[0])+x_train, x_test, y_train, y_test = (+    x[:train_indices],+    x[train_indices:],+    y[:train_indices],+    y[train_indices:],+)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+The model computes a score using dot product of user and movie embeddings.+Thereafter, it uses a sigmoid activation on the sum over all the scores (including biases).+Finally, the score is converted into the rating ranges based on this dataset.+"""+EMBEDDING_SIZE = 50+++class RecommenderNet(keras.Model):+    def __init__(+        self, num_users, num_movies, embedding_size, min_rating, max_rating, **kwargs+    ):+        super(RecommenderNet, self).__init__(**kwargs)+        self.num_users = num_users+        self.num_movies = num_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating

Remove

siddBanPsu

comment created time in a day

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: [Siddhartha Banerjee](https://twitter.com/sidd2006)+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using embedding-based model on Movielens dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering)+using [Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+The MovieLens ratings dataset includes users and movies with the corresponding ratings.+The goal is to be able to predict ratings for movies an user has not yet watched.+Thereafter, the movies with higher predicted ratings will be recommended to the user.++The steps in the model are as follows:++1. Feed user identifier and movie identifier to the model.+2. Use an embedding matrix to obtain the corresponding user and movie embeddings.+3. Obtain a dot product of the embeddings, the get a score and add bias.+4. Use a sigmoid activation to scale the ratings between 0-1 scale.+5. Convert the scaled output to a number between minimum and maximum rating.++**References:**++- [Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/371920.372071)+- [Neural Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/3038912.3052569)+"""++import pandas as pd+import numpy as np+from zipfile import ZipFile+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from pathlib import Path+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file+movielens_data_file_url = (+    "http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+)+movielens_zipped_file = keras.utils.get_file(+    "ml-latest-small.zip", movielens_data_file_url, extract=False+)+keras_datasets_path = Path(movielens_zipped_file).parents[0]+movielens_dir = keras_datasets_path / "ml-latest-small"++# Run the following piece only if it is the first time.+# Otherwise, data should already be there.+# In case the MovieLens 100k data has changed over time,+# then please delete the folder in datasets manually and rerun.+if not movielens_dir.exists():+    with ZipFile(movielens_zipped_file, "r") as zip:+        # extracting all the files+        print("Extracting all the files now...")+        zip.extractall(path=keras_datasets_path)+        print("Done!")++ratings_file = movielens_dir / "ratings.csv"+df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""+user_ids = df["userId"].unique().tolist()+user2user_encoded = {x: i for i, x in enumerate(user_ids)}+userencoded2user = {i: x for i, x in enumerate(user_ids)}+movie_ids = df["movieId"].unique().tolist()+movie2movie_encoded = {x: i for i, x in enumerate(movie_ids)}+movie_encoded2movie = {i: x for i, x in enumerate(movie_ids)}+df["user"] = df["userId"].map(user2user_encoded)+df["movie"] = df["movieId"].map(movie2movie_encoded)++num_users = len(user2user_encoded)+num_movies = len(movie_encoded2movie)+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        num_users, num_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""+df = df.sample(frac=1, random_state=42)+x = df[["user", "movie"]].values+# Normalize the targets between 0 and 1. Makes it easy to train.+y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values+# Assuming training on 90% of the data and validating on 10%.+train_indices = int(0.9 * df.shape[0])+x_train, x_test, y_train, y_test = (+    x[:train_indices],+    x[train_indices:],+    y[:train_indices],+    y[train_indices:],+)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+The model computes a score using dot product of user and movie embeddings.+Thereafter, it uses a sigmoid activation on the sum over all the scores (including biases).+Finally, the score is converted into the rating ranges based on this dataset.+"""+EMBEDDING_SIZE = 50+++class RecommenderNet(keras.Model):+    def __init__(+        self, num_users, num_movies, embedding_size, min_rating, max_rating, **kwargs+    ):+        super(RecommenderNet, self).__init__(**kwargs)+        self.num_users = num_users+        self.num_movies = num_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating

Remove

siddBanPsu

comment created time in a day

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: [Siddhartha Banerjee](https://twitter.com/sidd2006)+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using embedding-based model on Movielens dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering)+using [Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+The MovieLens ratings dataset includes users and movies with the corresponding ratings.+The goal is to be able to predict ratings for movies an user has not yet watched.+Thereafter, the movies with higher predicted ratings will be recommended to the user.++The steps in the model are as follows:++1. Feed user identifier and movie identifier to the model.+2. Use an embedding matrix to obtain the corresponding user and movie embeddings.+3. Obtain a dot product of the embeddings, the get a score and add bias.+4. Use a sigmoid activation to scale the ratings between 0-1 scale.+5. Convert the scaled output to a number between minimum and maximum rating.++**References:**++- [Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/371920.372071)+- [Neural Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/3038912.3052569)+"""++import pandas as pd+import numpy as np+from zipfile import ZipFile+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from pathlib import Path+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file+movielens_data_file_url = (+    "http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+)+movielens_zipped_file = keras.utils.get_file(+    "ml-latest-small.zip", movielens_data_file_url, extract=False+)+keras_datasets_path = Path(movielens_zipped_file).parents[0]+movielens_dir = keras_datasets_path / "ml-latest-small"++# Run the following piece only if it is the first time.+# Otherwise, data should already be there.+# In case the MovieLens 100k data has changed over time,+# then please delete the folder in datasets manually and rerun.+if not movielens_dir.exists():+    with ZipFile(movielens_zipped_file, "r") as zip:+        # extracting all the files+        print("Extracting all the files now...")+        zip.extractall(path=keras_datasets_path)+        print("Done!")++ratings_file = movielens_dir / "ratings.csv"+df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""+user_ids = df["userId"].unique().tolist()+user2user_encoded = {x: i for i, x in enumerate(user_ids)}+userencoded2user = {i: x for i, x in enumerate(user_ids)}+movie_ids = df["movieId"].unique().tolist()+movie2movie_encoded = {x: i for i, x in enumerate(movie_ids)}+movie_encoded2movie = {i: x for i, x in enumerate(movie_ids)}+df["user"] = df["userId"].map(user2user_encoded)+df["movie"] = df["movieId"].map(movie2movie_encoded)++num_users = len(user2user_encoded)+num_movies = len(movie_encoded2movie)+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        num_users, num_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""+df = df.sample(frac=1, random_state=42)+x = df[["user", "movie"]].values+# Normalize the targets between 0 and 1. Makes it easy to train.+y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values+# Assuming training on 90% of the data and validating on 10%.+train_indices = int(0.9 * df.shape[0])+x_train, x_test, y_train, y_test = (+    x[:train_indices],+    x[train_indices:],+    y[:train_indices],+    y[train_indices:],+)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+The model computes a score using dot product of user and movie embeddings.+Thereafter, it uses a sigmoid activation on the sum over all the scores (including biases).+Finally, the score is converted into the rating ranges based on this dataset.+"""+EMBEDDING_SIZE = 50+++class RecommenderNet(keras.Model):+    def __init__(+        self, num_users, num_movies, embedding_size, min_rating, max_rating, **kwargs+    ):+        super(RecommenderNet, self).__init__(**kwargs)+        self.num_users = num_users+        self.num_movies = num_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = layers.Embedding(+            num_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.user_bias = layers.Embedding(num_users, 1)+        self.movie_embedding = layers.Embedding(+            num_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.movie_bias = layers.Embedding(num_movies, 1)++    def call(self, inputs):+        user_vector = self.user_embedding(inputs[:, 0])+        user_bias = self.user_bias(inputs[:, 0])+        movie_vector = self.movie_embedding(inputs[:, 1])+        movie_bias = self.movie_bias(inputs[:, 1])+        dot_user_movie = layers.Dot(axes=1)([user_vector, movie_vector])

Why not use tf.tensordot here?

If you want to use Dot, you should instantiate the layer in __init__ and call it here (otherwise you are instantiating a new instance every time you call the model)

siddBanPsu

comment created time in a day

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: [Siddhartha Banerjee](https://twitter.com/sidd2006)+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using embedding-based model on Movielens dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering](https://en.wikipedia.org/wiki/Collaborative_filtering)+using [Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+The MovieLens ratings dataset includes users and movies with the corresponding ratings.+The goal is to be able to predict ratings for movies an user has not yet watched.+Thereafter, the movies with higher predicted ratings will be recommended to the user.++The steps in the model are as follows:++1. Feed user identifier and movie identifier to the model.+2. Use an embedding matrix to obtain the corresponding user and movie embeddings.+3. Obtain a dot product of the embeddings, the get a score and add bias.+4. Use a sigmoid activation to scale the ratings between 0-1 scale.+5. Convert the scaled output to a number between minimum and maximum rating.++**References:**++- [Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/371920.372071)+- [Neural Collaborative Filtering](https://dl.acm.org/doi/pdf/10.1145/3038912.3052569)+"""++import pandas as pd+import numpy as np+from zipfile import ZipFile+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from pathlib import Path+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file+movielens_data_file_url = (+    "http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+)+movielens_zipped_file = keras.utils.get_file(+    "ml-latest-small.zip", movielens_data_file_url, extract=False+)+keras_datasets_path = Path(movielens_zipped_file).parents[0]+movielens_dir = keras_datasets_path / "ml-latest-small"++# Run the following piece only if it is the first time.+# Otherwise, data should already be there.+# In case the MovieLens 100k data has changed over time,+# then please delete the folder in datasets manually and rerun.+if not movielens_dir.exists():+    with ZipFile(movielens_zipped_file, "r") as zip:+        # extracting all the files+        print("Extracting all the files now...")+        zip.extractall(path=keras_datasets_path)+        print("Done!")++ratings_file = movielens_dir / "ratings.csv"+df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""+user_ids = df["userId"].unique().tolist()+user2user_encoded = {x: i for i, x in enumerate(user_ids)}+userencoded2user = {i: x for i, x in enumerate(user_ids)}+movie_ids = df["movieId"].unique().tolist()+movie2movie_encoded = {x: i for i, x in enumerate(movie_ids)}+movie_encoded2movie = {i: x for i, x in enumerate(movie_ids)}+df["user"] = df["userId"].map(user2user_encoded)+df["movie"] = df["movieId"].map(movie2movie_encoded)++num_users = len(user2user_encoded)+num_movies = len(movie_encoded2movie)+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        num_users, num_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""+df = df.sample(frac=1, random_state=42)+x = df[["user", "movie"]].values+# Normalize the targets between 0 and 1. Makes it easy to train.+y = df["rating"].apply(lambda x: (x - min_rating) / (max_rating - min_rating)).values+# Assuming training on 90% of the data and validating on 10%.+train_indices = int(0.9 * df.shape[0])+x_train, x_test, y_train, y_test = (+    x[:train_indices],+    x[train_indices:],+    y[:train_indices],+    y[train_indices:],+)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+The model computes a score using dot product of user and movie embeddings.+Thereafter, it uses a sigmoid activation on the sum over all the scores (including biases).+Finally, the score is converted into the rating ranges based on this dataset.+"""+EMBEDDING_SIZE = 50+++class RecommenderNet(keras.Model):+    def __init__(+        self, num_users, num_movies, embedding_size, min_rating, max_rating, **kwargs+    ):+        super(RecommenderNet, self).__init__(**kwargs)+        self.num_users = num_users+        self.num_movies = num_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = layers.Embedding(+            num_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.user_bias = layers.Embedding(num_users, 1)+        self.movie_embedding = layers.Embedding(+            num_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=keras.regularizers.l2(1e-6),+        )+        self.movie_bias = layers.Embedding(num_movies, 1)++    def call(self, inputs):+        user_vector = self.user_embedding(inputs[:, 0])+        user_bias = self.user_bias(inputs[:, 0])+        movie_vector = self.movie_embedding(inputs[:, 1])+        movie_bias = self.movie_bias(inputs[:, 1])+        dot_user_movie = layers.Dot(axes=1)([user_vector, movie_vector])+        # Add all the components (including bias)+        x = dot_user_movie + user_bias + movie_bias+        # The sigmoid activation forces the rating to between 0 and 1+        x = tf.nn.sigmoid(x)

You can put the return on this line

siddBanPsu

comment created time in a day

push eventsiddBanPsu/keras-io

François Chollet

commit sha 50d3b237c56cf6ecc9000987c4d80cdf2e536e0e

Copy editing

view details

push time in a day

Pull request review commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

++**Author:** [ADMoreau](https://github.com/ADMoreau)<br>+**Date created:** 2020/05/17<br>+**Last modified:** 2020/05/23<br>+**Description:** PixelCNN implemented in Keras.+++<img class="k-inline-icon" src="https://colab.research.google.com/img/colab_favicon.ico"/> [**View in Colab**](https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/generative/ipynb/pixelcnn.ipynb)  <span class="k-dot">•</span><img class="k-inline-icon" src="https://github.com/favicon.ico"/> [**GitHub source**](https://github.com/keras-team/keras-io/blob/master/examples/generative/pixelcnn.py)++---+## Introduction++PixelCNN is a generative model proposed in 2016 by van den Oord et al.+(reference: [https://arxiv.org/abs/1606.05328](https://arxiv.org/abs/1606.05328)).+It is designed to generate images (or other data types) iteratively,+from an input vector where the probability distribution of prior elements dictates the+probability distribution of later elements. In the following example, images are generated+in this fashion, pixel-by-pixel, via a masked convolution kernel that only looks at data+from previously generated pixels (origin at the top left) to generate later pixels.+During inference, the output of the network is used as a probability ditribution+from which new pixel values are sampled to generate a new image+(here, with MNIST, the pixels values are either black or white).++++```python+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers++```++---+## Getting the Data++++```python+# Model / data parameters+num_classes = 10+input_shape = (28, 28, 1)+n_residual_blocks = 5+# The data, split between train and test sets+(x, _), (y, _) = keras.datasets.mnist.load_data()+# Concatenate all of the images together+data = np.concatenate((x, y), axis=0)+# Round all pixel values less than 33% of the max 256 value to 0+# anything above this value gets rounded up to 1 so that all values are either+# 0 or 1+data = np.where(data < (0.33 * 256), 0, 1)+data = data.astype(np.float32)++```++---+## Create two classes for the requisite Layers for the model++++```python+# The first layer is the PixelCNN layer. This layer simply+# builds on the 2D convolutional layer, but includes masking.+class PixelConvLayer(layers.Layer):+    def __init__(self, mask_type, **kwargs):+        super(PixelConvLayer, self).__init__()+        self.mask_type = mask_type+        self.conv = layers.Conv2D(**kwargs)++    def build(self, input_shape):+        # Build the conv2d layer to initialize kernel variables+        self.conv.build(input_shape)+        # Use the initialized kernel to create the mask+        kernel_shape = self.conv.kernel.get_shape()+        self.mask = np.zeros(shape=kernel_shape)+        self.mask[: kernel_shape[0] // 2, ...] = 1.0+        self.mask[kernel_shape[0] // 2, : kernel_shape[1] // 2, ...] = 1.0+        if self.mask_type == "B":+            self.mask[kernel_shape[0] // 2, kernel_shape[1] // 2, ...] = 1.0++    def call(self, inputs):+        self.conv.kernel.assign(self.conv.kernel * self.mask)+        return self.conv(inputs)+++# Next, we build our residual block layer.+# This is just a normal residual block, but base don the PixelConvLayer.+class ResidualBlock(keras.layers.Layer):+    def __init__(self, filters, **kwargs):+        super(ResidualBlock, self).__init__(**kwargs)+        self.activation = keras.layers.ReLU()+        self.conv1 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )+        self.pixel_conv = PixelConvLayer(+            mask_type="B",+            filters=filters // 2,+            kernel_size=3,+            activation="relu",+            padding="same",+        )+        self.conv2 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )++    def call(self, inputs):+        x = self.activation(inputs)+        x = self.conv1(x)+        x = self.pixel_conv(x)+        x = self.conv2(x)+        return keras.layers.add([inputs, x])+++```++---+## Build the model based on the original paper++++```python+inputs = keras.Input(shape=input_shape)+x = PixelConvLayer(mask_type="A", filters=128, kernel_size=7, padding="same")(inputs)++for _ in range(n_residual_blocks):+    x = ResidualBlock(filters=128)(x)++for _ in range(2):+    x = PixelConvLayer(+        mask_type="B",+        filters=128,+        kernel_size=1,+        strides=1,+        activation="relu",+        padding="valid",+    )(x)++out = keras.layers.Conv2D(+    filters=1, kernel_size=1, strides=1, activation="sigmoid", padding="valid"+)(x)++pixel_cnn = keras.Model(inputs, out)+adam = keras.optimizers.Adam(learning_rate=0.0001)+pixel_cnn.compile(optimizer=adam, loss="binary_crossentropy")++pixel_cnn.summary()+pixel_cnn.fit(x=data, y=data, batch_size=64, epochs=50, validation_split=0.1)++```++<div class="k-default-codeblock">+```+Model: "model"+_________________________________________________________________+Layer (type)                 Output Shape              Param #   +=================================================================+input_1 (InputLayer)         [(None, 28, 28, 1)]       0         +_________________________________________________________________+pixel_conv_layer (PixelConvL (None, 28, 28, 128)       6400      +_________________________________________________________________+residual_block (ResidualBloc (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_1 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_2 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_3 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+residual_block_4 (ResidualBl (None, 28, 28, 128)       98624     +_________________________________________________________________+pixel_conv_layer_6 (PixelCon (None, 28, 28, 128)       16512     +_________________________________________________________________+pixel_conv_layer_7 (PixelCon (None, 28, 28, 128)       16512     +_________________________________________________________________+conv2d_18 (Conv2D)           (None, 28, 28, 1)         129       +=================================================================+Total params: 532,673+Trainable params: 532,673+Non-trainable params: 0+_________________________________________________________________+Epoch 1/50+  2/985 [..............................] - ETA: 1:02 - loss: 0.6944WARNING:tensorflow:Callbacks method `on_train_batch_end` is slow compared to the batch time. Check your callbacks.

Make sure to train with the latest TF-nightly

ADMoreau

comment created time in a day

Pull request review commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

++**Author:** [ADMoreau](https://github.com/ADMoreau)<br>+**Date created:** 2020/05/17<br>+**Last modified:** 2020/05/23<br>+**Description:** PixelCNN implemented in Keras.+++<img class="k-inline-icon" src="https://colab.research.google.com/img/colab_favicon.ico"/> [**View in Colab**](https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/generative/ipynb/pixelcnn.ipynb)  <span class="k-dot">•</span><img class="k-inline-icon" src="https://github.com/favicon.ico"/> [**GitHub source**](https://github.com/keras-team/keras-io/blob/master/examples/generative/pixelcnn.py)++---+## Introduction++PixelCNN is a generative model proposed in 2016 by van den Oord et al.+(reference: [https://arxiv.org/abs/1606.05328](https://arxiv.org/abs/1606.05328)).+It is designed to generate images (or other data types) iteratively,+from an input vector where the probability distribution of prior elements dictates the+probability distribution of later elements. In the following example, images are generated+in this fashion, pixel-by-pixel, via a masked convolution kernel that only looks at data+from previously generated pixels (origin at the top left) to generate later pixels.+During inference, the output of the network is used as a probability ditribution+from which new pixel values are sampled to generate a new image+(here, with MNIST, the pixels values are either black or white).++++```python+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers++```++---+## Getting the Data++++```python+# Model / data parameters+num_classes = 10+input_shape = (28, 28, 1)+n_residual_blocks = 5+# The data, split between train and test sets+(x, _), (y, _) = keras.datasets.mnist.load_data()+# Concatenate all of the images together+data = np.concatenate((x, y), axis=0)+# Round all pixel values less than 33% of the max 256 value to 0+# anything above this value gets rounded up to 1 so that all values are either+# 0 or 1+data = np.where(data < (0.33 * 256), 0, 1)+data = data.astype(np.float32)++```++---+## Create two classes for the requisite Layers for the model++++```python+# The first layer is the PixelCNN layer. This layer simply+# builds on the 2D convolutional layer, but includes masking.+class PixelConvLayer(layers.Layer):+    def __init__(self, mask_type, **kwargs):+        super(PixelConvLayer, self).__init__()+        self.mask_type = mask_type+        self.conv = layers.Conv2D(**kwargs)++    def build(self, input_shape):+        # Build the conv2d layer to initialize kernel variables+        self.conv.build(input_shape)+        # Use the initialized kernel to create the mask+        kernel_shape = self.conv.kernel.get_shape()+        self.mask = np.zeros(shape=kernel_shape)+        self.mask[: kernel_shape[0] // 2, ...] = 1.0+        self.mask[kernel_shape[0] // 2, : kernel_shape[1] // 2, ...] = 1.0+        if self.mask_type == "B":+            self.mask[kernel_shape[0] // 2, kernel_shape[1] // 2, ...] = 1.0++    def call(self, inputs):+        self.conv.kernel.assign(self.conv.kernel * self.mask)+        return self.conv(inputs)+++# Next, we build our residual block layer.+# This is just a normal residual block, but base don the PixelConvLayer.

"based on"

ADMoreau

comment created time in a day

Pull request review commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

++**Author:** [ADMoreau](https://github.com/ADMoreau)<br>+**Date created:** 2020/05/17<br>+**Last modified:** 2020/05/23<br>+**Description:** PixelCNN implemented in Keras.+++<img class="k-inline-icon" src="https://colab.research.google.com/img/colab_favicon.ico"/> [**View in Colab**](https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/generative/ipynb/pixelcnn.ipynb)  <span class="k-dot">•</span><img class="k-inline-icon" src="https://github.com/favicon.ico"/> [**GitHub source**](https://github.com/keras-team/keras-io/blob/master/examples/generative/pixelcnn.py)++---+## Introduction++PixelCNN is a generative model proposed in 2016 by van den Oord et al.+(reference: [https://arxiv.org/abs/1606.05328](https://arxiv.org/abs/1606.05328)).+It is designed to generate images (or other data types) iteratively,+from an input vector where the probability distribution of prior elements dictates the+probability distribution of later elements. In the following example, images are generated+in this fashion, pixel-by-pixel, via a masked convolution kernel that only looks at data+from previously generated pixels (origin at the top left) to generate later pixels.+During inference, the output of the network is used as a probability ditribution+from which new pixel values are sampled to generate a new image+(here, with MNIST, the pixels values are either black or white).++++```python+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers++```++---+## Getting the Data++++```python+# Model / data parameters+num_classes = 10+input_shape = (28, 28, 1)+n_residual_blocks = 5+# The data, split between train and test sets+(x, _), (y, _) = keras.datasets.mnist.load_data()+# Concatenate all of the images together+data = np.concatenate((x, y), axis=0)+# Round all pixel values less than 33% of the max 256 value to 0+# anything above this value gets rounded up to 1 so that all values are either+# 0 or 1+data = np.where(data < (0.33 * 256), 0, 1)+data = data.astype(np.float32)++```++---+## Create two classes for the requisite Layers for the model++++```python+# The first layer is the PixelCNN layer. This layer simply+# builds on the 2D convolutional layer, but includes masking.+class PixelConvLayer(layers.Layer):+    def __init__(self, mask_type, **kwargs):+        super(PixelConvLayer, self).__init__()+        self.mask_type = mask_type+        self.conv = layers.Conv2D(**kwargs)++    def build(self, input_shape):+        # Build the conv2d layer to initialize kernel variables+        self.conv.build(input_shape)+        # Use the initialized kernel to create the mask+        kernel_shape = self.conv.kernel.get_shape()+        self.mask = np.zeros(shape=kernel_shape)+        self.mask[: kernel_shape[0] // 2, ...] = 1.0+        self.mask[kernel_shape[0] // 2, : kernel_shape[1] // 2, ...] = 1.0+        if self.mask_type == "B":+            self.mask[kernel_shape[0] // 2, kernel_shape[1] // 2, ...] = 1.0++    def call(self, inputs):+        self.conv.kernel.assign(self.conv.kernel * self.mask)+        return self.conv(inputs)+++# Next, we build our residual block layer.+# This is just a normal residual block, but base don the PixelConvLayer.+class ResidualBlock(keras.layers.Layer):+    def __init__(self, filters, **kwargs):+        super(ResidualBlock, self).__init__(**kwargs)+        self.activation = keras.layers.ReLU()

You don't appear to have removed this?

ADMoreau

comment created time in a day

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip

Actually, nevermind that comment, shell commands should be fine in this case.

dgriffiths3

comment created time in a day

pull request commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

Please update the PR whenever you have the time, and you can add the generated files. It's almost ready to merge :)

ADMoreau

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = K.eye(self.k)++    def __call__(self, x):+        x = K.reshape(x, (-1, self.k, self.k))+        xxt = K.batch_dot(x, x, axes=(2, 2))+        return K.sum(self.l2reg * K.square(xxt - self.eye))+++"""+ We can define the t-net as a functional model.+"""+++def tnet(n_pts, k, name):++    inputs = keras.Input(shape=(n_pts, k))++    # Initalise bias as the indentity matrix+    bias = keras.initializers.Constant(np.eye(k).flatten())+    reg = OrthogonalRegularizer(k)++    x = ConvBN(64)(inputs)+    x = ConvBN(128)(x)+    x = ConvBN(1024)(x)++    x = layers.GlobalMaxPooling1D()(x)++    x = DenseBN(512)(x)+    x = DenseBN(256)(x)+    x = layers.Dense(+        k * k,+        kernel_initializer="zeros",+        bias_initializer=bias,+        activity_regularizer=reg,+    )(x)++    feat_T = layers.Reshape((k, k))(x)++    # Apply affine transformation to input features+    outputs = layers.Dot(axes=(2, 1))([inputs, feat_T])++    return keras.Model(inputs=inputs, outputs=outputs, name=name)+++"""+The main network can be then implemented in the same manner where the t-net mini models+can be dropped in a layers in the graph. Here we replicate the network architecture+published in the original paper.+"""++inputs = keras.Input(shape=(N_PTS, 3))++# x = tnet(N_PTS, 3, 'input_tnet')(inputs)++x = ConvBN(64)(inputs)+x = ConvBN(64)(x)++# x = tnet(N_PTS, 64, 'feat_tnet')(x)

?

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = K.eye(self.k)++    def __call__(self, x):+        x = K.reshape(x, (-1, self.k, self.k))+        xxt = K.batch_dot(x, x, axes=(2, 2))+        return K.sum(self.l2reg * K.square(xxt - self.eye))+++"""+ We can define the t-net as a functional model.+"""+++def tnet(n_pts, k, name):++    inputs = keras.Input(shape=(n_pts, k))++    # Initalise bias as the indentity matrix+    bias = keras.initializers.Constant(np.eye(k).flatten())+    reg = OrthogonalRegularizer(k)++    x = ConvBN(64)(inputs)+    x = ConvBN(128)(x)+    x = ConvBN(1024)(x)++    x = layers.GlobalMaxPooling1D()(x)++    x = DenseBN(512)(x)+    x = DenseBN(256)(x)+    x = layers.Dense(+        k * k,+        kernel_initializer="zeros",+        bias_initializer=bias,+        activity_regularizer=reg,+    )(x)++    feat_T = layers.Reshape((k, k))(x)++    # Apply affine transformation to input features+    outputs = layers.Dot(axes=(2, 1))([inputs, feat_T])++    return keras.Model(inputs=inputs, outputs=outputs, name=name)+++"""+The main network can be then implemented in the same manner where the t-net mini models+can be dropped in a layers in the graph. Here we replicate the network architecture+published in the original paper.+"""++inputs = keras.Input(shape=(N_PTS, 3))++# x = tnet(N_PTS, 3, 'input_tnet')(inputs)++x = ConvBN(64)(inputs)+x = ConvBN(64)(x)++# x = tnet(N_PTS, 64, 'feat_tnet')(x)++x = ConvBN(64)(x)+x = ConvBN(128)(x)+x = ConvBN(1024)(x)++x = layers.GlobalMaxPooling1D()(x)+

You can trim a lot of blank space, in general

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = K.eye(self.k)++    def __call__(self, x):+        x = K.reshape(x, (-1, self.k, self.k))+        xxt = K.batch_dot(x, x, axes=(2, 2))+        return K.sum(self.l2reg * K.square(xxt - self.eye))+++"""+ We can define the t-net as a functional model.+"""+++def tnet(n_pts, k, name):++    inputs = keras.Input(shape=(n_pts, k))++    # Initalise bias as the indentity matrix+    bias = keras.initializers.Constant(np.eye(k).flatten())+    reg = OrthogonalRegularizer(k)++    x = ConvBN(64)(inputs)+    x = ConvBN(128)(x)+    x = ConvBN(1024)(x)++    x = layers.GlobalMaxPooling1D()(x)++    x = DenseBN(512)(x)+    x = DenseBN(256)(x)+    x = layers.Dense(+        k * k,+        kernel_initializer="zeros",+        bias_initializer=bias,+        activity_regularizer=reg,+    )(x)++    feat_T = layers.Reshape((k, k))(x)++    # Apply affine transformation to input features+    outputs = layers.Dot(axes=(2, 1))([inputs, feat_T])++    return keras.Model(inputs=inputs, outputs=outputs, name=name)+++"""+The main network can be then implemented in the same manner where the t-net mini models+can be dropped in a layers in the graph. Here we replicate the network architecture+published in the original paper.+"""++inputs = keras.Input(shape=(N_PTS, 3))++# x = tnet(N_PTS, 3, 'input_tnet')(inputs)++x = ConvBN(64)(inputs)+x = ConvBN(64)(x)++# x = tnet(N_PTS, 64, 'feat_tnet')(x)++x = ConvBN(64)(x)+x = ConvBN(128)(x)+x = ConvBN(1024)(x)++x = layers.GlobalMaxPooling1D()(x)++x = DenseBN(512)(x)+x = layers.Dropout(0.3)(x)++x = DenseBN(256)(x)+x = layers.Dropout(0.3)(x)++outputs = layers.Dense(NUM_CLASSES, activation="softmax")(x)++model = keras.Model(inputs=inputs, outputs=outputs, name="pointnet")++model.summary()++"""+### Train model+"""+

Remove blank space

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = K.eye(self.k)++    def __call__(self, x):+        x = K.reshape(x, (-1, self.k, self.k))+        xxt = K.batch_dot(x, x, axes=(2, 2))+        return K.sum(self.l2reg * K.square(xxt - self.eye))+++"""+ We can define the t-net as a functional model.+"""+++def tnet(n_pts, k, name):++    inputs = keras.Input(shape=(n_pts, k))++    # Initalise bias as the indentity matrix+    bias = keras.initializers.Constant(np.eye(k).flatten())+    reg = OrthogonalRegularizer(k)++    x = ConvBN(64)(inputs)+    x = ConvBN(128)(x)+    x = ConvBN(1024)(x)++    x = layers.GlobalMaxPooling1D()(x)++    x = DenseBN(512)(x)+    x = DenseBN(256)(x)+    x = layers.Dense(+        k * k,+        kernel_initializer="zeros",+        bias_initializer=bias,+        activity_regularizer=reg,+    )(x)++    feat_T = layers.Reshape((k, k))(x)++    # Apply affine transformation to input features+    outputs = layers.Dot(axes=(2, 1))([inputs, feat_T])++    return keras.Model(inputs=inputs, outputs=outputs, name=name)+++"""+The main network can be then implemented in the same manner where the t-net mini models+can be dropped in a layers in the graph. Here we replicate the network architecture+published in the original paper.+"""++inputs = keras.Input(shape=(N_PTS, 3))++# x = tnet(N_PTS, 3, 'input_tnet')(inputs)++x = ConvBN(64)(inputs)+x = ConvBN(64)(x)++# x = tnet(N_PTS, 64, 'feat_tnet')(x)++x = ConvBN(64)(x)+x = ConvBN(128)(x)+x = ConvBN(1024)(x)++x = layers.GlobalMaxPooling1D()(x)++x = DenseBN(512)(x)+x = layers.Dropout(0.3)(x)++x = DenseBN(256)(x)+x = layers.Dropout(0.3)(x)++outputs = layers.Dense(NUM_CLASSES, activation="softmax")(x)++model = keras.Model(inputs=inputs, outputs=outputs, name="pointnet")++model.summary()++"""+### Train model+"""++"""+Once the model is defined it can be trained like any other standard classification model+using `.compile()` and `.fit()`.+"""++model.compile(+    loss="sparse_categorical_crossentropy",+    optimizer=keras.optimizers.Adam(learning_rate=0.001),+    metrics=["sparse_categorical_accuracy"],+)++model.fit(train_dataset, epochs=10, validation_data=test_dataset)

After training, it would help to visualize some predictions. How long does training take on a K80?

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = K.eye(self.k)++    def __call__(self, x):+        x = K.reshape(x, (-1, self.k, self.k))+        xxt = K.batch_dot(x, x, axes=(2, 2))+        return K.sum(self.l2reg * K.square(xxt - self.eye))+++"""+ We can define the t-net as a functional model.+"""+++def tnet(n_pts, k, name):++    inputs = keras.Input(shape=(n_pts, k))++    # Initalise bias as the indentity matrix+    bias = keras.initializers.Constant(np.eye(k).flatten())+    reg = OrthogonalRegularizer(k)++    x = ConvBN(64)(inputs)+    x = ConvBN(128)(x)+    x = ConvBN(1024)(x)++    x = layers.GlobalMaxPooling1D()(x)++    x = DenseBN(512)(x)+    x = DenseBN(256)(x)+    x = layers.Dense(+        k * k,+        kernel_initializer="zeros",+        bias_initializer=bias,+        activity_regularizer=reg,+    )(x)++    feat_T = layers.Reshape((k, k))(x)++    # Apply affine transformation to input features+    outputs = layers.Dot(axes=(2, 1))([inputs, feat_T])++    return keras.Model(inputs=inputs, outputs=outputs, name=name)+++"""+The main network can be then implemented in the same manner where the t-net mini models+can be dropped in a layers in the graph. Here we replicate the network architecture+published in the original paper.+"""++inputs = keras.Input(shape=(N_PTS, 3))++# x = tnet(N_PTS, 3, 'input_tnet')(inputs)

?

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):+        self.k = k+        self.l2reg = l2reg+        self.eye = K.eye(self.k)++    def __call__(self, x):+        x = K.reshape(x, (-1, self.k, self.k))

Prefer using tf.reshape, tf.tensordot, tf.reduce_sum. Don't use backend ops

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical+representation. The second is an affine transformation for alignment in feature space ($n+\times 64$). As per the original paper we constrain the transformation to be close to an+orthogonal matrix (i.e. $||X*X^T - I||$ = 0).+"""+++class OrthogonalRegularizer(keras.regularizers.Regularizer):+    def __init__(self, k, l2reg=0.001):

Please add a docstring and use more descriptive argument names

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048

NUM_POINTS, etc

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini+network. The mini network strongly resembles the main network. The t-net is used twice.+The first time to transform the input features ($n \times 3$) into a canonical

Do not use LaTeX notation since we don't render it. Instead, just in-line Python code or pseudo code.

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):+    def __init__(self, filters):+        super(ConvBN, self).__init__()++        self.conv = layers.Conv1D(filters, kernel_size=1, padding="valid")+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.conv(inputs)+        x = self.bn(x)+        return self.activation(x)+++class DenseBN(layers.Layer):+    def __init__(self, filters):+        super(DenseBN, self).__init__()++        self.dense = layers.Dense(filters)+        self.bn = layers.BatchNormalization(momentum=0.0)+        self.activation = tf.nn.relu++    def call(self, inputs):++        x = self.dense(inputs)+        x = self.bn(x)+        return self.activation(x)+++"""+PointNet consists of two core components. The primary MLP network, and the transformer+net (t-net). The t-net aims to learn an affine transformation matrix by its own mini

T-Net

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.+"""+++class ConvBN(layers.Layer):

A functional approach is much simpler:

def conv_bn(x, filters):
   x = layers.Conv1D(filters, kernel_size=1, padding="valid")(x)
   x = layers.BatchNormalization(momentum=0.0)(x)
   return layers.Activation('relu')(x)

Same for DenseBN

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""++"""+Each convolution and fully-connected layer (with exception for end layers) consits of+Convolution / Dense -> Batch Normalization -> ReLU Activation. We can make simple+sub-classed layer to perform all these as a single layer.

You don't need to, you can take a fully functional approach, which is more concise and does not create additional deserialization overhead

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))+test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))++train_dataset = train_dataset.shuffle(len(X_train)).map(augment).batch(BATCH_SIZE)+test_dataset = test_dataset.shuffle(len(X_test)).batch(BATCH_SIZE)++"""+### Build a model+"""+

Remove blank space

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):++    # jitter points+    pts += tf.random.uniform(pts.shape, -0.005, 0.005, dtype=tf.float64)+    # shuffle points+    pts = tf.random.shuffle(pts)+    # random rotation about the z axis+    theta = np.random.uniform(-np.pi, np.pi)+    c = np.cos(theta)+    s = np.sin(theta)+    R = tf.constant([[c, -s, 0.0], [s, c, 0.0], [0.0, 0.0, 1.0]], tf.float64)+    pts = tf.einsum("...ij,...kj->...ki", R, pts)++    return pts, label+++train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))

Before generating the dataset, it would be useful to have a visualization of what one set of points looks like.

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)++"""+Our data can now be read into a `tf.data.Dataset()` object. We set the shuffle buffer+size to the entire size of the dataset as prior to this the data is ordered by class.+Data augmentation is important when working with point cloud data. We create a+augmentation function to jitter, shuffle and rotate the train dataset.+"""+++def augment(pts, label):+

Remove blank line

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):++        print("processing: %s" % folder)+        train_files = glob.glob(folder + "/train/*")+        test_files = glob.glob(folder + "/test/*")++        for f in train_files:+            train_pts.append(trimesh.load(f).sample(n_pts))+            train_labels.append(i)++        for f in test_files:+            test_pts.append(trimesh.load(f).sample(n_pts))+            test_labels.append(i)++    return (+        np.array(train_pts),+        np.array(test_pts),+        np.array(train_labels),+        np.array(test_labels),+    )+++"""+Set the number of points to sample and batch size and parse the dataset. This can take+~5minutes to complete.+"""++N_PTS = 2048+NUM_CLASSES = 10+BATCH_SIZE = 32++X_train, X_test, y_train, y_test = parse_dataset(N_PTS)

Use lowercase for variable names, e.g. x_train

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):+

Remove blank line

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []+    train_labels = []+    test_pts = []+    test_labels = []++    folders = glob.glob("ModelNet10/[!README]*")++    for i, folder in enumerate(folders):+

Remove blank line

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):++    train_pts = []

train_points, etc.

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip

Prefer using get_file to download and cache the file, and unzip it in Python. Having shell commands means the script can't run as a standalone Python file, so shell commands should only be used when there's no alternative.

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""++"""+We use the ModelNet10 model dataset, the smaller 10 class version of the ModelNet40+dataset. First download the data:+"""++"""shell+!curl -O http://3dvision.princeton.edu/projects/2014/3DShapeNets/ModelNet10.zip+!unzip ModelNet10.zip+"""++"""+We can use the `trimesh` package to read and visualize the `.off` mesh files.+"""++mesh = trimesh.load("ModelNet10/chair/train/chair_0001.off")+mesh.show()++"""+To convert a mesh file to a point cloud we first need to sample points on the mesh+surface. `.sample()` performs a unifrom random sampling. Here we sample at 2048 locations+and visualize in `matplotlib`.+"""++points = mesh.sample(2048)+trimesh.points.plot_points(points)++"""+To generate a `tf.data.Dataset()` we need to first parse through the ModelNet data+folders. Each mesh is loaded and sampled into a point cloud before being added to a+standard python list and converted to a `numpy` array. We also store the current+enumerate index value as the object label.+"""+++def parse_dataset(n_pts=2048):

Prefer using fully spelled-out variable names, like num_points

dgriffiths3

comment created time in 2 days

Pull request review commentkeras-team/keras-io

PointNet example

+"""+Title: Point cloud classification with PointNet+Author: [David Griffiths](https://dgriffiths3.github.io)+Date created: 2020/05/25+Last modified: 2020/05/25+Description: Implementation of PointNet for ModelNet10 classification.+"""++"""+## Introduction+"""++"""+Classification, detection and segmentation of unordered 3D point sets i.e. point clouds+is a core problem in computer vision. This example implements the seminal point cloud+deep learning paper [PointNet (Qi et al., 2017)](https://arxiv.org/abs/1612.00593). For a+detailed intoduction on PointNet see [this blog+post](https://medium.com/@luis_gonzales/an-in-depth-look-at-pointnet-111d7efdaa1a).+"""++"""+## Setup+"""++"""+If running in google colab first run `!pip install trimesh`.+"""++import glob+import trimesh+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tensorflow.keras import backend as K++"""+## Load dataset+"""+

No need for the separation between these 2 text blocks

dgriffiths3

comment created time in 2 days

push eventkeras-team/keras-io

François Chollet

commit sha c98310e5c9ee1cf6b55bd5736bb823934f38a594

Sort code examples by conceptual reading order

view details

push time in 2 days

pull request commentkeras-team/keras-io

Deep Q Network example

Looks great! I applied some minor style nits.

Could you please add a gif rendering of the trained network playing the game at the end?

How long does it take the train on a K80?

jacoblchapman

comment created time in 2 days

push eventjacoblchapman/keras-io

François Chollet

commit sha 7cb5a56315d82971b39afb8b66efa2afe78c1169

Minor style nits

view details

push time in 2 days

Pull request review commentkeras-team/autokeras

enable and tested saving of categorical to numerical layer

 def get_config(self):  CUSTOM_OBJECTS = {     'MultiColumnCategoricalEncoding': MultiColumnCategoricalEncoding,+    'IndexLookup': index_lookup.IndexLookup,

Consider using StringLookup and IntegerLookup respectively, instead of IndexLookup. The latter will not be exposed in the API

haifeng-jin

comment created time in 2 days

push eventkeras-team/keras-io

François Chollet

commit sha 3c01ead8d47fe06aefb3df55f0f0aee94de7c472

Fix typo

view details

push time in 2 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)+        x = Lambda(lambda x: x * (self.max_rating - self.min_rating) + self.min_rating)(

Better to scale the target ratings than the network's outputs (it's a lot less work and it makes the loss range more typical)

siddBanPsu

comment created time in 2 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)+        x = Lambda(lambda x: x * (self.max_rating - self.min_rating) + self.min_rating)(+            x+        )+        return x+++inputs = layers.Input(shape=(2,))+recommendation_layer = RecommenderNet(+    n_users, n_movies, EMBEDDING_SIZE, min_rating, max_rating+)+outputs = recommendation_layer(inputs)+model = keras.Model(inputs=inputs, outputs=outputs)+opt = Adam(lr=0.001)+model.compile(loss="mean_squared_error", optimizer=opt)

Try it out -- regression done with NNs often works better with crossentropy.

siddBanPsu

comment created time in 2 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)+        x = Lambda(lambda x: x * (self.max_rating - self.min_rating) + self.min_rating)(+            x+        )+        return x+++inputs = layers.Input(shape=(2,))+recommendation_layer = RecommenderNet(+    n_users, n_movies, EMBEDDING_SIZE, min_rating, max_rating+)+outputs = recommendation_layer(inputs)+model = keras.Model(inputs=inputs, outputs=outputs)+opt = Adam(lr=0.001)+model.compile(loss="mean_squared_error", optimizer=opt)

Why use MSE instead of binary crossentropy (assuming that you normalize the movie ratings between 0 and 1)?

siddBanPsu

comment created time in 2 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)+        x = Lambda(lambda x: x * (self.max_rating - self.min_rating) + self.min_rating)(

Can you explain the purpose of this line?

siddBanPsu

comment created time in 2 days

Pull request review commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

+"""+Title: PixelCNN+Author: [ADMoreau](https://github.com/ADMoreau)+Date created: 2020/05/17+Last modified: 2020/05/23+Description: PixelCNN implemented in Keras.+"""++"""+## Introduction+PixelCNN is a generative model proposed in 2016 by van den Oord et al.+(https://arxiv.org/abs/1606.05328). It is designed to generate images or other data types+from an input vector where the probability distribution of prior elements dictate the+probability distribution of later elements. In the following example images are generated+in this fashion through a masked convolutional kernel that is only capable of using data+from earlier pixels (origin at the top left) to generate later pixels. During inference,+the output of the network is used as a probability ditribution from which pixel values+for the desired generated image are sampled (here, with MNIST, the pixels values are+either black or white) from the network output.+"""++import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers++"""+##Getting the Data+"""++# Model / data parameters+num_classes = 10+input_shape = (28, 28, 1)+n_residual_blocks = 5+# the data, split between train and test sets+(x, _), (y, _) = keras.datasets.mnist.load_data()+# Concatenate all of the images together+data = np.concatenate((x, y), axis=0)+# round all pixel values less than 33% of the max 256 value to 0+# anything above this value gets rounded up to 1 so that all values are either+# 0 or 1+data = np.where(data < (0.33 * 256), 0, 1)+data = data.astype(np.float32)++"""+## Create two classes for the requisite Layers for the model+"""++# The first layer is the PixelCNN layer. This layer simply+# builds on the 2D convolutional layer, but includes masking.+class PixelConvLayer(layers.Layer):+    def __init__(self, mask_type, **kwargs):+        super(PixelConvLayer, self).__init__()+        self.mask_type = mask_type+        self.conv = layers.Conv2D(**kwargs)++    def build(self, input_shape):+        # Build the conv2d layer to initialize kernel variables+        self.conv.build(input_shape)+        # Use the initialized kernel to create the mask+        kernel_shape = self.conv.kernel.get_shape()+        self.mask = np.zeros(shape=kernel_shape)+        self.mask[: kernel_shape[0] // 2, ...] = 1.0+        self.mask[kernel_shape[0] // 2, : kernel_shape[1] // 2, ...] = 1.0+        if self.mask_type == "B":+            self.mask[kernel_shape[0] // 2, kernel_shape[1] // 2, ...] = 1.0++    def call(self, inputs):+        self.conv.kernel.assign(self.conv.kernel * self.mask)+        return self.conv(inputs)+++# Next, we build our residual block layer.+# This is just a normal residual block, but base don the PixelConvLayer.+class ResidualBlock(keras.layers.Layer):+    def __init__(self, filters, **kwargs):+        super(ResidualBlock, self).__init__(**kwargs)+        self.activation = keras.layers.ReLU()+        self.conv1 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )+        self.pixel_conv = PixelConvLayer(+            mask_type="B",+            filters=filters // 2,+            kernel_size=3,+            activation="relu",+            padding="same",+        )+        self.conv2 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )++    def call(self, inputs):+        x = self.activation(inputs)+        x = self.conv1(x)+        x = self.pixel_conv(x)+        x = self.conv2(x)+        return keras.layers.add([inputs, x])+++"""+## Build the model based on the original paper+"""++inputs = keras.Input(shape=input_shape)+x = PixelConvLayer(+    mask_type="A", filters=128, kernel_size=7, padding="same", activation="relu"+)(inputs)++for _ in range(n_residual_blocks):+    x = ResidualBlock(filters=128)(x)

Both parts of the addition appear to be already relu-ed (one part is the input, which is always relu-ed before the block, and the other is the output of a conv-relu). So it should be safe to remove.

ADMoreau

comment created time in 3 days

Pull request review commentkeras-team/autokeras

fixing prediction with dataset for multi io model

 def _prepare_data(self, x, y, validation_data, validation_split):                 validation_split)         return dataset, validation_data +    def _get_x(self, dataset):+        shapes = data_utils.dataset_shape(dataset)+        # Only one or less element in the first level.+        print(len(shapes))

Leftover from debugging?

haifeng-jin

comment created time in 3 days

Pull request review commentkeras-team/autokeras

fixing prediction with dataset for multi io model

 def _prepare_data(self, x, y, validation_data, validation_split):                 validation_split)         return dataset, validation_data +    def _get_x(self, dataset):

This method would benefit form a docstring, since the name is not self-explanatory

haifeng-jin

comment created time in 3 days

push eventkeras-team/keras-io

François Chollet

commit sha e2dcdd47ecb363cbcf66e2091b1a0767824a850c

Update text classification from scratch example

view details

push time in 3 days

push eventkeras-team/keras-io

François Chollet

commit sha 941013ba2cbf314e23a8b0fa933bcaa94dbaf4e4

Trim logs for text extraction example

view details

push time in 3 days

push eventkeras-team/keras-io

François Chollet

commit sha eb411256cce8e36a246d9b2089ec07ca4d9e74f9

Remove extra blank line

view details

push time in 3 days

push eventkeras-team/keras-io

Apoorv Nandan

commit sha 3eedad19ae8e0848490853a15bad26e7a539f9bf

Added example for fine tuning BERT on Text Extraction task (SQuAD) (#46) * Added actor critic example * Added comments * Annotated code and folder rename * Black reformat * Style nits * Added visualizations * black reformat * Added ipynb and md files * Delete tutobooks-checkpoint.py * Added bert text extraction example * Style nits * Updated comments and callback * Added ipynb and md files * nits * added comment on number of epochs Co-authored-by: Apoorv Nandan <apoorv.nandan@gmail.com> Co-authored-by: François Chollet <francois.chollet@gmail.com>

view details

push time in 3 days

PR merged keras-team/keras-io

Added example for fine tuning BERT on Text Extraction task (SQuAD)
  • Uses pretrained bert-base-uncased from HuggingFace and their tokenizers.
  • Gets an exact match score of 79.4 after 2 epoch on TPU. (Paper shows 80.8, most likely due to a bunch of extra post-processing steps that I saw in the original implementation while converting the predicted token indexes back to the answer text)
  • We will have to limit the data before training begins because this will take hours on CPU.
+1395 -0

4 comments

3 changed files

apoorvnandan

pr closed time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2

For these 2 imports, just from tensorflow import keras and then use e.g. keras.regularizers.l2

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values

Use lowercase variable names, e.g. x_train

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)+        x = Lambda(lambda x: x * (self.max_rating - self.min_rating) + self.min_rating)(+            x+        )+        return x+++inputs = layers.Input(shape=(2,))+recommendation_layer = RecommenderNet(+    n_users, n_movies, EMBEDDING_SIZE, min_rating, max_rating+)+outputs = recommendation_layer(inputs)+model = keras.Model(inputs=inputs, outputs=outputs)+opt = Adam(lr=0.001)+model.compile(loss="mean_squared_error", optimizer=opt)+model.summary()

You won't be able to print the summary of a subclassed model

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.

Please add a quick explanation of the model, and maybe links to external references about collaborative filtering.

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder

You also don't need sklearn here, you can do it by hand in a couple of lines (see e.g. the credit card fraud detection example)

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"

Instead, use keras.utils.get_file to download the file and retrieve its path, then use zipfile in Python to unzip it

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split

You don't need sklearn to do this

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (

Do not import individual layers, instead just import from tensorflow.keras import layers

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)+        x = Lambda(lambda x: x * (self.max_rating - self.min_rating) + self.min_rating)(+            x+        )+        return x+++inputs = layers.Input(shape=(2,))+recommendation_layer = RecommenderNet(+    n_users, n_movies, EMBEDDING_SIZE, min_rating, max_rating+)+outputs = recommendation_layer(inputs)+model = keras.Model(inputs=inputs, outputs=outputs)

RecommenderNet subclasses Model, so you can just do

model = RecommenderNet(...)

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])

Use tf.tensordot instead

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):

You don't seem to use this layer anywhere, so remove it. In general, you don't need to create a layer subclass for something like this, since it's a simple combination of 2 built-in layers.

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users

Use num_* as number prefix (same below).

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")

Don't provide a hard-coded value as the name (it will create name conflicts if you create multiple instances as part of a single model), instead accept **kwargs in the signature and pass them to super

Note that the name will default to recommender_net, which is fine.

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)

x = tf.nn.sigmoid(x)

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,

Use num_* as number prefix.

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])

Prefer using fully spelled-out variable names, such as "user_vectors"

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])

x = x + ub + mb

siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added script to perform collaborative filtering on 100k Movielens dataset

+"""+Title: Collaborative Filtering for Movie Recommendations+Author: Siddhartha Banerjee+Date created: 2020/05/24+Last modified: 2020/05/24+Description: Recommending movies using user and movie embeddings trained using Movielens small dataset.+"""+"""+## Introduction++This example looks at+[Collaborative filtering using Movielens dataset](https://www.kaggle.com/c/movielens-100k)+to recommend movies to users.+"""++import tensorflow as tf+import pandas as pd+import numpy as np+from tensorflow import keras+from pathlib import Path+from tensorflow.keras import layers+from sklearn.model_selection import train_test_split+from sklearn.preprocessing import LabelEncoder+from tensorflow.keras.models import Model+from tensorflow.keras.layers import (+    Input,+    Reshape,+    Dot,+    Add,+    Embedding,+    Activation,+    Lambda,+    Dense,+)+from tensorflow.keras.optimizers import Adam+from tensorflow.keras.regularizers import l2+import matplotlib.pyplot as plt++"""+## First, load the data and apply some pre-processing+"""++# Download the actual data from http://files.grouplens.org/datasets/movielens/ml-latest-small.zip"+# use the ratings.csv file++data_directory = Path("/Users/siddban/Documents/ml-latest-small/")++ratings_file = data_directory / "ratings.csv"++df = pd.read_csv(ratings_file)++"""+First, need to perform some preprocessing to encode users and movies to specific indices.+This is done as the movieids and userids are non-sequential in nature.+"""++user_enc = LabelEncoder()+df["user"] = user_enc.fit_transform(df["userId"].values)+n_users = df["user"].nunique()+movie_enc = LabelEncoder()+df["movie"] = movie_enc.fit_transform(df["movieId"].values)+n_movies = df["movie"].nunique()+df["rating"] = df["rating"].values.astype(np.float32)+min_rating = min(df["rating"])+max_rating = max(df["rating"])+print(+    "Number of users: {}, Number of Movies: {}, Min rating: {}, Max rating: {}".format(+        n_users, n_movies, min_rating, max_rating+    )+)++"""+## Prepare training and validation data+"""++X = df[["user", "movie"]].values+y = df["rating"].values+X_train, X_test, y_train, y_test = train_test_split(+    X, y, test_size=0.1, random_state=42+)+print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)++"""+## Create the model++Let's fix some parameters in the architecture. We will use user embeddings and movie+embeddings, let's keep the dimensions as 50 for both.+"""+EMBEDDING_SIZE = 50+++class EmbeddingLayer(layers.Layer):+    def __init__(self, n_items, n_factors):+        super(EmbeddingLayer, self).__init__()+        self.n_items = n_items+        self.n_factors = n_factors+        self.emb = Embedding(+            self.n_items,+            self.n_factors,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )++    def call(self, x):+        x = self.emb(x)+        x = Reshape((self.n_factors,))(x)+        return x+++class RecommenderNet(keras.Model):+    def __init__(self, n_users, n_movies, embedding_size, min_rating, max_rating):+        super(RecommenderNet, self).__init__(name="recnet")+        self.n_users = n_users+        self.n_movies = n_movies+        self.embedding_size = embedding_size+        self.min_rating = min_rating+        self.max_rating = max_rating+        self.user_embedding = Embedding(+            n_users,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.user_bias = Embedding(n_users, 1)+        self.movie_embedding = Embedding(+            n_movies,+            embedding_size,+            embeddings_initializer="he_normal",+            embeddings_regularizer=l2(1e-6),+        )+        self.movie_bias = Embedding(n_movies, 1)++    def call(self, inputs):+        u = self.user_embedding(inputs[:, 0])+        ub = self.user_bias(inputs[:, 0])+        m = self.movie_embedding(inputs[:, 1])+        mb = self.movie_bias(inputs[:, 1])+        x = Dot(axes=1)([u, m])+        x = Add()([x, ub, mb])+        x = Activation("sigmoid")(x)+        x = Lambda(lambda x: x * (self.max_rating - self.min_rating) + self.min_rating)(

You don't need to use a Lambda layer, you can just in-line the code:

x = x * (self.max_rating - self.min_rating) + self.min_rating
siddBanPsu

comment created time in 3 days

Pull request review commentkeras-team/keras-io

Added example for fine tuning BERT on Text Extraction task (SQuAD)

++# BERT (from HuggingFace Transformers) for Text Extraction++**Author:** [Apoorv Nandan](https://twitter.com/NandanApoorv)<br>+**Date created:** 2020/05/23<br>+**Last modified:** 2020/05/23<br>+++<img class="k-inline-icon" src="https://colab.research.google.com/img/colab_favicon.ico"/> [**View in Colab**](https://colab.research.google.com/github/keras-team/keras-io/blob/master/examples/nlp/ipynb/text_extraction_with_bert.ipynb)  <span class="k-dot">•</span><img class="k-inline-icon" src="https://github.com/favicon.ico"/> [**GitHub source**](https://github.com/keras-team/keras-io/blob/master/examples/nlp/text_extraction_with_bert.py)+++**Description:** Fine tune pretrained BERT from HuggingFace Transformers on SQuAD.++---+## Introduction++This demonstration uses SQuAD (Stanford Question-Answering Dataset).+In SQuAD, an input consists of a question, and a paragraph for context.+The goal is to find the span of text in the paragraph that answers the question.+We evaluate our performance on this data with the "Exact Match" metric,+which measures the percentage of predictions that exactly match any one of the+ground-truth answers.++We fine-tune a BERT model to perform this task as follows:++1. Feed the context and the question as inputs to BERT.+2. Take two vectors S and T with dimensions equal to that of+   hidden states in BERT.+3. Compute the probability of each token being the start and end of+   the answer span. The probability of a token being the start of+   the answer is given by a dot product between S and the representation+   of the token in the last layer of BERT, followed by a softmax over all tokens.+   The probability of a token being the end of the answer is computed+   similarly with the vector T.+4. Fine-tune BERT and learn S and T along the way.++References:+- [BERT](https://arxiv.org/pdf/1810.04805.pdf)+- [SQuAD](https://arxiv.org/abs/1606.05250)+++Setup++++```python+import os+import re+import json+import string+import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers+from tokenizers import BertWordPieceTokenizer+from transformers import BertTokenizer, TFBertModel, BertConfig++max_len = 384+configuration = BertConfig()  # default paramters and configuration for BERT++```++---+## Set-up BERT tokenizer++++```python+# Save the slow pretrained tokenizer+slow_tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")+save_path = "bert_base_uncased/"+if not os.path.exists(save_path):+    os.makedirs(save_path)+slow_tokenizer.save_pretrained(save_path)++# Load the fast tokenizer from saved file+tokenizer = BertWordPieceTokenizer("bert_base_uncased/vocab.txt", lowercase=True)++```++---+## Load the data++++```python+train_data_url = "https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json"+train_path = keras.utils.get_file("train.json", train_data_url)+eval_data_url = "https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json"+eval_path = keras.utils.get_file("eval.json", eval_data_url)++```++---+## Preprocess the data++1. Go through the JSON file and store every record as a `SquadExample` object.+2. Go through each `SquadExample` and create `X_train, y_train, X_eval, y_eval`.++++```python++class SquadExample:+    def __init__(self, question, context, start_char_idx, answer_text, all_answers):+        self.question = question+        self.context = context+        self.start_char_idx = start_char_idx+        self.answer_text = answer_text+        self.all_answers = all_answers+        self.skip = False++    def preprocess(self):+        context = self.context+        question = self.question+        answer_text = self.answer_text+        start_char_idx = self.start_char_idx++        # Clean context, answer and question+        context = " ".join(str(context).split())+        question = " ".join(str(question).split())+        answer = " ".join(str(answer_text).split())++        # Find end character index of answer in context+        end_char_idx = start_char_idx + len(answer)+        if end_char_idx >= len(context):+            self.skip = True+            return++        # Mark the character indexes in context that are in answer+        is_char_in_ans = [0] * len(context)+        for idx in range(start_char_idx, end_char_idx):+            is_char_in_ans[idx] = 1++        # Tokenize context+        tokenized_context = tokenizer.encode(context)++        # Find tokens that were created from answer characters+        ans_token_idx = []+        for idx, (start, end) in enumerate(tokenized_context.offsets):+            if sum(is_char_in_ans[start:end]) > 0:+                ans_token_idx.append(idx)++        if len(ans_token_idx) == 0:+            self.skip = True+            return++        # Find start and end token index for tokens from answer+        start_token_idx = ans_token_idx[0]+        end_token_idx = ans_token_idx[-1]++        # Tokenize question+        tokenized_question = tokenizer.encode(question)++        # Create inputs+        input_ids = tokenized_context.ids + tokenized_question.ids[1:]+        token_type_ids = [0] * len(tokenized_context.ids) + [1] * len(+            tokenized_question.ids[1:]+        )+        attention_mask = [1] * len(input_ids)++        # Pad and create attention masks.+        # Skip if truncation is needed+        padding_length = max_len - len(input_ids)+        if padding_length > 0:  # pad+            input_ids = input_ids + ([0] * padding_length)+            attention_mask = attention_mask + ([0] * padding_length)+            token_type_ids = token_type_ids + ([0] * padding_length)+        elif padding_length < 0:  # skip+            self.skip = True+            return++        self.input_ids = input_ids+        self.token_type_ids = token_type_ids+        self.attention_mask = attention_mask+        self.start_token_idx = start_token_idx+        self.end_token_idx = end_token_idx+        self.context_token_to_char = tokenized_context.offsets+++with open(train_path) as f:+    raw_train_data = json.load(f)++with open(eval_path) as f:+    raw_eval_data = json.load(f)+++def create_squad_examples(raw_data):+    squad_examples = []+    for item in raw_data["data"]:+        for para in item["paragraphs"]:+            context = para["context"]+            for qa in para["qas"]:+                question = qa["question"]+                answer_text = qa["answers"][0]["text"]+                all_answers = [_["text"] for _ in qa["answers"]]+                start_char_idx = qa["answers"][0]["answer_start"]+                squad_eg = SquadExample(+                    question, context, start_char_idx, answer_text, all_answers+                )+                squad_eg.preprocess()+                squad_examples.append(squad_eg)+    return squad_examples+++def create_inputs_targets(squad_examples):+    dataset_dict = {+        "input_ids": [],+        "token_type_ids": [],+        "attention_mask": [],+        "start_token_idx": [],+        "end_token_idx": [],+    }+    for item in squad_examples:+        if item.skip == False:+            for key in dataset_dict:+                dataset_dict[key].append(getattr(item, key))+    for key in dataset_dict:+        dataset_dict[key] = np.array(dataset_dict[key])++    x = [+        dataset_dict["input_ids"],+        dataset_dict["token_type_ids"],+        dataset_dict["attention_mask"],+    ]+    y = [dataset_dict["start_token_idx"], dataset_dict["end_token_idx"]]+    return x, y+++train_squad_examples = create_squad_examples(raw_train_data)+x_train, y_train = create_inputs_targets(train_squad_examples)+print(f"{len(train_squad_examples)} training points created.")++eval_squad_examples = create_squad_examples(raw_eval_data)+x_eval, y_eval = create_inputs_targets(eval_squad_examples)+print(f"{len(eval_squad_examples)} evaluation points created.")++```++<div class="k-default-codeblock">+```+87599 training points created.+10570 evaluation points created.++```+</div>+Create the Question-Answering Model using BERT and Functional API++++```python++def create_model():+    ## BERT encoder+    encoder = TFBertModel.from_pretrained("bert-base-uncased")++    ## QA Model+    input_ids = layers.Input(shape=(max_len,), dtype=tf.int32)+    token_type_ids = layers.Input(shape=(max_len,), dtype=tf.int32)+    attention_mask = layers.Input(shape=(max_len,), dtype=tf.int32)+    embedding = encoder(+        input_ids, token_type_ids=token_type_ids, attention_mask=attention_mask+    )[0]++    start_logits = layers.Dense(1, name="start_logit", use_bias=False)(embedding)+    start_logits = layers.Flatten()(start_logits)++    end_logits = layers.Dense(1, name="end_logit", use_bias=False)(embedding)+    end_logits = layers.Flatten()(end_logits)++    start_probs = layers.Activation(keras.activations.softmax)(start_logits)+    end_probs = layers.Activation(keras.activations.softmax)(end_logits)++    model = keras.Model(+        inputs=[input_ids, token_type_ids, attention_mask],+        outputs=[start_probs, end_probs],+    )+    loss = keras.losses.SparseCategoricalCrossentropy(from_logits=False)+    optimizer = keras.optimizers.Adam(lr=5e-5)+    model.compile(optimizer=optimizer, loss=[loss, loss])+    return model+++```++This code should preferably be run on Google Colab TPU runtime.+With Colab TPUs, each epoch will take 5-6 minutes.++++```python+use_tpu = True+if use_tpu:+    # Create distribution strategy+    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()+    tf.config.experimental_connect_to_cluster(tpu)+    tf.tpu.experimental.initialize_tpu_system(tpu)+    strategy = tf.distribute.experimental.TPUStrategy(tpu)++    # Create model+    with strategy.scope():+        model = create_model()+else:+    model = create_model()++model.summary()++```++<div class="k-default-codeblock">+```+INFO:absl:Entering into master device scope: /job:worker/replica:0/task:0/device:CPU:0++INFO:tensorflow:Initializing the TPU system: grpc://10.48.159.170:8470++INFO:tensorflow:Initializing the TPU system: grpc://10.48.159.170:8470++INFO:tensorflow:Clearing out eager caches++INFO:tensorflow:Clearing out eager caches++INFO:tensorflow:Finished initializing TPU system.++INFO:tensorflow:Finished initializing TPU system.++INFO:tensorflow:Found TPU system:++INFO:tensorflow:Found TPU system:++INFO:tensorflow:*** Num TPU Cores: 8++INFO:tensorflow:*** Num TPU Cores: 8++INFO:tensorflow:*** Num TPU Workers: 1++INFO:tensorflow:*** Num TPU Workers: 1++INFO:tensorflow:*** Num TPU Cores Per Worker: 8++INFO:tensorflow:*** Num TPU Cores Per Worker: 8++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0)++INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0)++Model: "model"+__________________________________________________________________________________________________+Layer (type)                    Output Shape         Param #     Connected to                     +==================================================================================================+input_1 (InputLayer)            [(None, 384)]        0                                            +__________________________________________________________________________________________________+input_3 (InputLayer)            [(None, 384)]        0                                            +__________________________________________________________________________________________________+input_2 (InputLayer)            [(None, 384)]        0                                            +__________________________________________________________________________________________________+tf_bert_model (TFBertModel)     ((None, 384, 768), ( 109482240   input_1[0][0]                    +__________________________________________________________________________________________________+start_logit (Dense)             (None, 384, 1)       768         tf_bert_model[0][0]              +__________________________________________________________________________________________________+end_logit (Dense)               (None, 384, 1)       768         tf_bert_model[0][0]              +__________________________________________________________________________________________________+flatten (Flatten)               (None, 384)          0           start_logit[0][0]                +__________________________________________________________________________________________________+flatten_1 (Flatten)             (None, 384)          0           end_logit[0][0]                  +__________________________________________________________________________________________________+activation_7 (Activation)       (None, 384)          0           flatten[0][0]                    +__________________________________________________________________________________________________+activation_8 (Activation)       (None, 384)          0           flatten_1[0][0]                  +==================================================================================================+Total params: 109,483,776+Trainable params: 109,483,776+Non-trainable params: 0+__________________________________________________________________________________________________++```+</div>+---+## Create evaluation Callback++This callback will compute the exact match score using the validation data+after every epoch.++++```python++def normalize_text(text):+    text = text.lower()++    # Remove punctuations+    exclude = set(string.punctuation)+    text = "".join(ch for ch in text if ch not in exclude)++    # Remove articles+    regex = re.compile(r"\b(a|an|the)\b", re.UNICODE)+    text = re.sub(regex, " ", text)++    # Remove extra white space+    text = " ".join(text.split())+    return text+++class ExactMatch(keras.callbacks.Callback):+    """+    Each `SquadExample` object contains the character level offsets for each token+    in its input paragraph. We use them to get back the span of text corresponding+    to the tokens between our predicted start and end tokens.+    All the ground-truth answers are also present in each `SquadExample` object.+    We calculate the percentage of data points where the span of text obtained+    from model predictions matches one of the ground-truth answers.+    """++    def __init__(self, x_eval, y_eval):+        self.x_eval = x_eval+        self.y_eval = y_eval++    def on_epoch_end(self, epoch, logs=None):+        pred_start, pred_end = self.model.predict(self.x_eval)+        count = 0+        eval_examples_no_skip = [_ for _ in eval_squad_examples if _.skip == False]+        for idx, (start, end) in enumerate(zip(pred_start, pred_end)):+            squad_eg = eval_examples_no_skip[idx]+            offsets = squad_eg.context_token_to_char+            start = np.argmax(start)+            end = np.argmax(end)+            if start >= len(offsets):+                continue+            pred_char_start = offsets[start][0]+            if end < len(offsets):+                pred_char_end = offsets[end][1]+                pred_ans = squad_eg.context[pred_char_start:pred_char_end]+            else:+                pred_ans = squad_eg.context[pred_char_start:]++            normalized_pred_ans = normalize_text(pred_ans)+            normalized_true_ans = [normalize_text(_) for _ in squad_eg.all_answers]+            if normalized_pred_ans in normalized_true_ans:+                count += 1+        acc = count / len(self.y_eval[0])+        print(f"\nepoch={epoch+1}, exact match score={acc:.2f}")+++```++---+## Train and Evaluate++++```python+exact_match_callback = ExactMatch(x_eval, y_eval)+model.fit(+    x_train,+    y_train,+    epochs=1,

Please add a comment here that the recommend number of epochs is 3, not 1. You don't need to regenerate the files, you can juste edit the ipynb and md files directly to add the comment.

apoorvnandan

comment created time in 3 days

Pull request review commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

+"""+Title: PixelCNN+Author: [ADMoreau](https://github.com/ADMoreau)+Date created: 2020/05/17+Last modified: 2020/05/23+Description: PixelCNN implemented in Keras.+"""++"""+## Introduction+PixelCNN is a generative model proposed in 2016 by van den Oord et al.+(https://arxiv.org/abs/1606.05328). It is designed to generate images or other data types+from an input vector where the probability distribution of prior elements dictate the+probability distribution of later elements. In the following example images are generated+in this fashion through a masked convolutional kernel that is only capable of using data+from earlier pixels (origin at the top left) to generate later pixels. During inference,+the output of the network is used as a probability ditribution from which pixel values+for the desired generated image are sampled (here, with MNIST, the pixels values are+either black or white) from the network output.+"""++import numpy as np+import tensorflow as tf+from tensorflow import keras+from tensorflow.keras import layers++"""+##Getting the Data+"""++# Model / data parameters+num_classes = 10+input_shape = (28, 28, 1)+n_residual_blocks = 5+# the data, split between train and test sets+(x, _), (y, _) = keras.datasets.mnist.load_data()+# Concatenate all of the images together+data = np.concatenate((x, y), axis=0)+# round all pixel values less than 33% of the max 256 value to 0+# anything above this value gets rounded up to 1 so that all values are either+# 0 or 1+data = np.where(data < (0.33 * 256), 0, 1)+data = data.astype(np.float32)++"""+## Create two classes for the requisite Layers for the model+"""++# The first layer is the PixelCNN layer. This layer simply+# builds on the 2D convolutional layer, but includes masking.+class PixelConvLayer(layers.Layer):+    def __init__(self, mask_type, **kwargs):+        super(PixelConvLayer, self).__init__()+        self.mask_type = mask_type+        self.conv = layers.Conv2D(**kwargs)++    def build(self, input_shape):+        # Build the conv2d layer to initialize kernel variables+        self.conv.build(input_shape)+        # Use the initialized kernel to create the mask+        kernel_shape = self.conv.kernel.get_shape()+        self.mask = np.zeros(shape=kernel_shape)+        self.mask[: kernel_shape[0] // 2, ...] = 1.0+        self.mask[kernel_shape[0] // 2, : kernel_shape[1] // 2, ...] = 1.0+        if self.mask_type == "B":+            self.mask[kernel_shape[0] // 2, kernel_shape[1] // 2, ...] = 1.0++    def call(self, inputs):+        self.conv.kernel.assign(self.conv.kernel * self.mask)+        return self.conv(inputs)+++# Next, we build our residual block layer.+# This is just a normal residual block, but base don the PixelConvLayer.+class ResidualBlock(keras.layers.Layer):+    def __init__(self, filters, **kwargs):+        super(ResidualBlock, self).__init__(**kwargs)+        self.activation = keras.layers.ReLU()+        self.conv1 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )+        self.pixel_conv = PixelConvLayer(+            mask_type="B",+            filters=filters // 2,+            kernel_size=3,+            activation="relu",+            padding="same",+        )+        self.conv2 = keras.layers.Conv2D(+            filters=filters, kernel_size=1, activation="relu"+        )++    def call(self, inputs):+        x = self.activation(inputs)+        x = self.conv1(x)+        x = self.pixel_conv(x)+        x = self.conv2(x)+        return keras.layers.add([inputs, x])+++"""+## Build the model based on the original paper+"""++inputs = keras.Input(shape=input_shape)+x = PixelConvLayer(+    mask_type="A", filters=128, kernel_size=7, padding="same", activation="relu"+)(inputs)++for _ in range(n_residual_blocks):+    x = ResidualBlock(filters=128)(x)

The initial input to this block is already relu-ed (by the layer at line 107), and then every subsequent input got relu-ed by the last conv2 layer in the block. So it appears that the ReLU layer at the start of the block is not doing any work here. Unless I'm missing something?

ADMoreau

comment created time in 3 days

push eventADMoreau/keras-io

François Chollet

commit sha e897a1f3f50e2d20d36f5772c96877ec97891128

Style nits

view details

push time in 3 days

issue closedkeras-team/keras-io

Update logs for VAE example for overriding train_step

I am learning how to override train_step from the guides. It seems to me that instead of returning the values here: https://github.com/keras-team/keras-io/blob/master/examples/generative/vae.py#L91

We need to update some metrics like this:

self.loss_metric.update_state(total_loss)
self.reconstruction_loss_metric.update_state(reconstruction_loss)
self.kl_loss_metric.update_state(kl_loss)

return {
            "loss": total_loss,
            "reconstruction_loss": reconstruction_loss,
            "kl_loss": kl_loss,
        }

I might be wrong, but I think if we do not do this, the model.fit with verbose option just prints the loss for the final batch of the epoch instead of the average loss on the whole epoch. I guess it would be great if test_step is also added to and explained in these guides.

closed time in 3 days

siavash-khodadadeh

push eventkeras-team/keras-io

François Chollet

commit sha a15366045a1f3969fc109b81f8f6b33187c39abc

Fix typo

view details

push time in 3 days

pull request commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

It seems the current branch is no longer up-to-date (e.g. changes in the residual block are gone). Please make sure the Python in the PR is up-to-date. Also, note that you should not include the autogenerated files (md file, ipynb file, and image files) until the PR has been approved.

ADMoreau

comment created time in 3 days

pull request commentkeras-team/keras-io

Added example for fine tuning BERT on Text Extraction task (SQuAD)

How long does it take to run the example on a V100?

apoorvnandan

comment created time in 4 days

pull request commentkeras-team/keras-io

added PixelCNN example, added pathlib to requirements

Looks good! Please merge back this commit (changes got reverted) and apply the black style formatter.

ADMoreau

comment created time in 4 days

more