Utility files for human pose estimation in python
jackd/adversarially_parameterized_optimization 14
GAN-based 3D human pose estimation model for 3DV'17 paper
Cython iterative farthest point sampling implementation
Dictionary Interface to DataSets for python
ANTLR4 runtime library for Dart.
Tensorflow implementation of CrfRnn layer
A collection of datasets ready to use with TensorFlow
tensorflow_datasets implementations for events datasets
template deformation network using tensorflow 2.0 and keras
Finite volume method for simulating fluid flow and associated processes in porous media
pull request commenttensorflow/datasets
[updated PR] added mpii human pose dataset
@googlebot I consent
comment created time in 15 days
pull request commenttensorflow/datasets
[updated PR] added mpii human pose dataset
@hermannsblum go for it, I never worked out what tfds needed to get this through and I'm working on very different stuff now.
comment created time in 15 days
issue openedtensorflow/tensorflow
keras.Model.fit cannot handle variable epoch sizes
<em>Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template</em>
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu 18.04
- TensorFlow installed from (source or binary): binary
- TensorFlow version (use command below): v2.2.0-0-g2b96f3662b 2.2.0
- Python version: 3.6.9
Describe the current behavior
keras.Model.fit with finite-length dataset inputs can run without steps_per_epoch. However, if the number of batches in the epoch changes between epochs, this can lead to:
- truncation of epoch if subsequence epochs are longer than the initial epoch; and
- truncation of training if subsequence epochsa re shorter than the initial epoch.
While not a common occurance, this happens e.g. in graph neural network applications where it makes more sense to batch according to some maximum number of edges/nodes in a graph as opposed to a fixed batch size. A fixed number of graphs in a dataset can result in a variable number of batches when these are shuffled.
Describe the expected behavior
Each epoch of fit should iterate over a dataset in it's entirety when steps_per_epoch is not provided, and run to the defined number of epochs (unless terminated prior with callbacks), even if the number of steps varies between epochs. Alternatively, a special steps_per_epoch value (say, -1) should be able to be provided to specify a dynamic epoch size.
Standalone code to reproduce the issue
Colab
Other info / logs From colab above, when subsequence epochs are shorter than the first:
WARNING:tensorflow:Your input ran out of data; interrupting training. Make sure that your dataset or generator can generate at least `steps_per_epoch * epochs` batches (in this case, 30 batches). You may need to use the repeat() function when building your dataset.
created time in a month
push eventjackd/kblocks
commit sha ee71f6a7ec97b4a084149e79ba20ab151d144eed
added license
push time in 2 months
issue commentjackd/more-keras
Hi @Evizero, my latest iteration of this is kblocks. I'll throw an MIT license up over there now...
comment created time in 2 months
push eventjackd/tensorflow
commit sha 57eccc7bc29ddb105dcaa2f6a413163461ad9987
added test
push time in 2 months
push eventjackd/numba-neighbors
commit sha 9145e057384951f5869eb7b99cfbbc644bbf0311
Add explicit license
commit sha c8eca4f177e7e39b8dfc77869b0f3b51ab91121c
Merge pull request #1 from Huite/master Add explicit MIT license to repo
push time in 2 months
PR merged jackd/numba-neighbors
Very minor PR, mostly an excuse to get in touch!
I was looking at some KDTree implementations, especially one that works well with numba. I found your comment in this issue: https://github.com/numba/numba-scipy/issues/36 ... which obviously brought me here!
Some background: I'm coming from a geospatial use case, I want to do some (2D for now) unstructured mesh regridding which involves computing some areal overlaps from overlapping cells. Most (vector-based) GIS stuff seems awfully slow or some existing solutions come with massive binary dependencies, so I figured I'd have a look at it myself. A KDTree with a radius query seems like a decent starting point to create some sort of short list of possibly overlapping cells. I greatly prefer numba since it's basically seamless to Python, and it's much nicer to distribute than e.g. Cython; the JIT'ing also allows arbritary area weighting functions at runtime, so it's great all around.
Anyway, numba-neighbors is looking pretty spiffy! I've been looking at these:
- https://github.com/storpipfugl/pykdtree
- https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.cKDTree.html
- https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KDTree.html
And for my ad hoc benchmark (but fairly realistic for the use case), numba-neighbors beats them by a fairly wide margin (> 30% on cKTree, more so on the others). Might be interesting to redo this analysis at some point: https://jakevdp.github.io/blog/2013/04/29/benchmarking-nearest-neighbor-searches-in-python/
Anyway, my main issue with the other methods is how they return the indices. I don't really want to be stuck with a fixed number of neighbors, so sklearn's radius_query is what I want... except it returns an array of arrays which probably isn't going to be great for further numba functions to work on. I could copy and edit some Cython stuff, but as mentioned numba is just much nicer to distribute. So with numba-neighbors it should be pretty easy to write a custom query function for my goals!
Wrapping up: numba-neigbors seems useful, I'd like to use it. I saw you did put license='MIT' in the setup.py -- but I had to look for it (if only briefly).
pr closed time in 2 months
pull request commentjackd/numba-neighbors
Add explicit MIT license to repo
Thanks for getting in touch / fixing this up - glad to hear you're finding it useful. This was very much a "I have code I developed for another project, might as well open source it" rather than "let's build a general-purpose KDTree library", and I opened it up in a bit of a rush. There'll no doubt be other things like this, and happy to hear thoughts on the interface. I'm a little tied up with other things at the moment to give this a thorough cleaning, but if you notice anything else dodgy/confusing I'll keep an eye on PRs :).
So with
numba-neighborsit should be pretty easy to write a custom query function for my goals!
Note the BinaryTree class has a query_radius method. It's fairly similar to those other interfaces, except
(a) you need to specify a max_count for pre-allocation purposes (and if you under-estimate the returned indices are not necessarily the closest); and
(b) it doesn't sort the returned indices by distance.
I originally wrote wrappers that solved each of these issues, but they slowed things down and weren't necessary for the use case I developed it for.
comment created time in 2 months
issue commentjackd/pointnet_ops
Do you have write access to pointnet_ops directory?
comment created time in 2 months
PR opened tensorflow/tensorflow
In particular, empty tuples should not trigger this.
pr created time in 2 months
push eventjackd/tensorflow
commit sha 82143c1ad88ceeb51ac6a280b79c2bc766dd854b
Enable TensorCore embeddings for training via FeatureColumnV2. PiperOrigin-RevId: 312340625 Change-Id: I559aba797a8f1a37ecec1e4ee71cd027701ae6dd
commit sha deaf5ea06a618749113f7def6285ebf77c19dfa7
Addressed PR comments
commit sha 935c55c590898f589de230c60ccbc6d50f09a8c7
Fix performance regression involving trainable check in batchnorm. The regression made it so a tf `and` op would be used for something that just required a python check, which in turn would make an if statement build a tf.cond instead of a python if. This change makes it just use a python if. PiperOrigin-RevId: 312345759 Change-Id: I568c9c992287bfc3e693f34b7b51bd7f35388f34
commit sha dc0149adb93c612f2c1f7fb6fc294ebcfee8e36a
Test ConvertActivation in dynamic shape mode Additionally helper functions for dynamic shape tests are improved: - Added test parameter member variables to ParameterizedOpConverterTestBase - Single TetOpConverter function introduced - Type parameter handling for input tensors simplified
commit sha baa3e80ca55909ce1b56864c0adcd825862fea63
Fix pylint errors PiperOrigin-RevId: 312356181 Change-Id: I7259f4e5584e2947dcf919918a5e9371e50185fe
commit sha 53215ab702aedb306778590277f71768c2d2c148
Use the nonlocal mechanism for if statements. This is the same mechanism used by for and while loops and it allows reusing much of the code. This required the ternary if operator to be split in a separate implementation, but that better accounts for its different nature. This should also allow more consistent verification and error messages throughout. PiperOrigin-RevId: 312360755 Change-Id: I57989c6cd40a16653521e18ccf21f2b0e994bd96
commit sha af2263101b0407ff1fb7f7e492565f1edc4cee30
Don't try t gather from empty tensors PiperOrigin-RevId: 312361331 Change-Id: I8f81add090d9a5452e671c48a03e0f5cb9f81a41
commit sha ca53894d61ca46e3d6a007a6de0c8c3458ead931
Canonicalize tf.Select to tf.SelectV2. The ops are mostly equivalent, except that Select has stricter requirements and does not support broadcasting, whereas SelectV2 does. There is one special case to be considered in this canonicalization, which is when the predicate is a tensor and the data is multidimensional. In this case, Select op semantics dictate that the predicate tensor length must match the size of the first data dimension. This varies from normal broadcasting semantics, which are used in SelectV2, so we must reshape the tensor in this case to be compatible. This also adds verifiers and tests for the Select and SelectV2 ops in the MLIR TF dialect. PiperOrigin-RevId: 312362580 Change-Id: I43f326ad330c92ce279b25cecf5a2cf46714ce3f
commit sha 930709e46e3b80345ffeac92b7728873dc97b0a7
Hexagon delegate: support SLICE op for uint8/int8 PiperOrigin-RevId: 312362625 Change-Id: Idf0185c33bc7d64cf480a70166c15f64b0a409b1
commit sha d894109fe1203f2259819841b85a0354c7780609
Fix parameter check for batchdim in Gather. PiperOrigin-RevId: 312371119 Change-Id: I7537194147199136b5b847ce6d1ddd361e42a393
commit sha 91da977a0305f2c25c6c149a5924d8b1eb33375c
Improve the bucket generation in Boosted Trees to avoid returning more than requested buckets. PiperOrigin-RevId: 312371738 Change-Id: I7f241c839f52d679ad4ceb82c161018e9b944fa3
commit sha 119aa03c7605a82061227c3291e8c5665752c90f
Split delegate-specific interpreter tests into a separate file PiperOrigin-RevId: 312372505 Change-Id: If366a884ce090f2ad40bdc20d266ef32eb5a1765
commit sha f0eb6dff6fb4f0500f45c8ca2b82c365de17f403
adjust gather ops launch config. for NCF model, this means ~20% gain. (due to grid size from 80->160 on volta). PiperOrigin-RevId: 312373706 Change-Id: I2413d301ec170e6e90eeae025e4bb17fccd5abbb
commit sha db573482f4f8712ff173f9bb511dceef4128228c
Add a folder for numpy API PiperOrigin-RevId: 312374580 Change-Id: Ic8be81b738659668814e956d7fd4da7972944257
commit sha e5fcb88fa425f6c905d5bbb28d0da2bfa6257587
Fixed sanity test issue
commit sha f8a918ccf6d39aa6c1dbf56716f1bd476322c1cd
Allow tile op to work on variant dtype. PiperOrigin-RevId: 312382133 Change-Id: I3a0f95865ca0f782fa73f7ba55b3d987de006332
commit sha c9819edcfc30424c52e24c4288ccf9fd7df30b63
pfor: Enable handling VariableShape PiperOrigin-RevId: 312382248 Change-Id: Ie8f600dadeb6f5ef3bb7483f7435348ac002e176
commit sha 6fad0820f83b8d90fb33acad4589563b479dbd73
Use enum instead of string to specify the context type. PiperOrigin-RevId: 312385361 Change-Id: I7ee9c203e7f662bf6898d80d9fe2b75536d0e044
commit sha 692bb1da53493a6cf37dc28a4c1e1a82df32d9fa
Fix nightly breakage (macos, linux). PiperOrigin-RevId: 312388059 Change-Id: I9db7d5e73b82298df3f800ece69cdccd92e706ca
commit sha 7f3ef3e1eae4d7142cb3b52ede78caa18a37e96c
[Profiler] Add optimization advice for reducing the host-to-TPU data transfer time. PiperOrigin-RevId: 312388184 Change-Id: I2fc8a60af6724467e447026dde7a8d6925ed1357
push time in 2 months
pull request commenttensorflow/community
@penpornk is there a reason this isn't exposed as a CompositeTensor?
Separately, can I request the following ops are considered/exposed in python?
- constructor that takes components (
values, column_indices, row_pointers, batch_pointers) - properties that return components
CSRMatrix.with_values(new_values)that creates a new matrix with the same sparsity but withvaluesreplaced withnew_values, similar toRaggedTensor.with_values
comment created time in 2 months
pull request commenttensorflow/community
@Padarn tf-graphics has some use-cases with graph neural networks, though they're currently implemented with tf.gather + tf.reduce_sum . Performance benefits using COO are inconsistent. See this PR for equivalent implementations and the discussion for rough benchmarks.
comment created time in 2 months
pull request commenttensorflow/community
Would there be benefit in implementing indices as a RaggedTensor, rather than col_indices, row_pointers, batch_pointers? I'm not sure if RaggedTensors are a thing in the c++ API, but I imagine this would make a lot of things easier in the python API, e.g. batching?
comment created time in 3 months
issue openedcgtuebingen/Flex-Convolution
I found the supplementary material for the paper a while back, but can't seem to find it any more. Is it still available somewhere?
created time in 3 months
PR closed tensorflow/graphics
Small adjustments to pointnet data pipeline.
A couple of things that will make a difference to performance under certain configurations:
- fixed bug with point cropping (original cropping was done on the batch dimension)
- removed data augmentation from test set, even when data augmentation is turned on (as per original implementation).
Optimizations:
- moved preprocessing ops into
tf.data.Dataset.map - added
prefetch
I don't imagine these optimizations will provide an observable difference in this case, but I think it is appropriate nonetheless if this is to be used as an example.
pr closed time in 3 months
pull request commenttensorflow/graphics
I like to have a clear separation between data stored on disk (from tfds) and preprocessing with tf.data, e.g. I don't think it's appropriate for jittering to be a part of the builder/config itself, but perhaps a preprocessing file alongside it. I guess it depends on whether or not you feel preprocessing is a property of the data or part of the model. I tend to feel its part of the model (though even then it's not clear cut. e.g. I feel the number of points is a property of the the data, while the jittering is a property of the model).
comment created time in 3 months
pull request commenttensorflow/graphics
optimized/debugged data pipeline
@ataiya if #312 isn't too big to do in one PR I might abandon this one - I've already resolved merge conflicts there, will address your comments there shortly. Happy to do both PRs if necessary, just don't want to waste your time.
comment created time in 3 months
pull request commenttensorflow/graphics
@ataiya I'm happy to duplicate learning rate / preprocessing across examples. The only reason I'm tempted to keep at least data preprocessing separate is for reusability, e.g. it's nice to be able to say with confidence "we used exactly the same data preprocessing as pointnet" without having to copy/paste from examples. I agree helpers.py wouldn't be the appropriate place for that, but perhaps a small preprocessing.py file?
comment created time in 3 months
Pull request review commenttensorflow/graphics
__all__.remove("version") # pylint: enable=g-statement-before-imports,g-import-not-at-top -__version__ = "HEAD"
I was wondering where I accidentally managed to change that...
comment created time in 3 months
push eventjackd/graphics
commit sha cccb4f320a0e4aa40a10ea0135e43bab34a55579
undid version change
push time in 3 months
issue commenttensorflow/graphics
Use TFDS mock_data for pointnet_test
@ataiya PR is merged, but I can't close.
comment created time in 3 months
push eventjackd/graphics
commit sha ed7173eccc5cbb6296da27557f9ca782f52f20e8
Updates TF Graphics metadata PiperOrigin-RevId: 304701489
commit sha df90fed7eb95b540783286167c3b9e0fc60be371
added metadata dir for tfds.mock_data
commit sha edab69c2101bf1899255c708ac6dfb5093fa8da0
relaxed logging to empty directory
commit sha 7b81d466570f094a486abb4b7c29d5b1cac54a5e
Added training/evaluation testing with mock_data
commit sha 837e742834f91a739838c44274c9f80626c75c83
fixed sys.argv
commit sha 28816e1f37ea2d552f873f00b07ead4cc26dfd41
moved mock_data to test
commit sha 078cac65c647e48c061555adf8d27046e86d1837
fix evaluation at end, not per step
commit sha 1a36e902d578d787d0674c92caa1b042ae40bf45
addressing comments
commit sha c2f100875432cd2609eb7c0d89e80aae0f103539
reformat, remove unused tfds flag
commit sha 55d48eac2a35092281bf85636151e8685869fa6e
formatting clargs
commit sha dbf8fb761652a0f7e062a0b2320624724f085e50
Deployed github workflow (path fix) PiperOrigin-RevId: 310838670
commit sha 8f17880b49a0a5e458b825b843e20e4322889d01
typo
commit sha 84195a0c6146f94a3f7eae395230e61121ed8eb4
Add Shapenet (ShapeNetCore.v2) dataset to tensorflow_datasets. PiperOrigin-RevId: 310935648
commit sha af7f2f9ddeeaa81e18cbf7f86b2778729bc3e17b
Fix to workflow PiperOrigin-RevId: 310980875
commit sha 5d902d033e9f0e3c8fce820eaa95f180881d3bc9
Exposes eps argument for tfg.math.cartesian_to_spherical_coordinates. PiperOrigin-RevId: 311384397
commit sha 0236a7f95d4f15d93da1081f7aafeb5609440dd7
Fix the travis widget to reflect the status of master. PiperOrigin-RevId: 311482619
commit sha 6800317e8095a07ffb915f482a122959e2adb3f3
Merge pull request #222 from tensorflow:test_304701489 PiperOrigin-RevId: 311745638
commit sha 5d35e72a8844b99192510402d2e3a7c7157663fb
Refactor the Shapenet dataset to support different BUILDER_CONFIGS. PiperOrigin-RevId: 311760835
commit sha 2a759c0981372cc58e51e77ba939b00693cb9fdf
Updated travis to avoid internal force-push
commit sha 87b673582aee03c63d32e6667fc76b1ff1dadaa4
Update to auto-documentation workflow. PiperOrigin-RevId: 311987485
push time in 3 months
pull request commenttensorflow/graphics
@julienvalentin is that the final review stage? Am I right to merge (or squash and merge)? I don't want to break anything...
comment created time in 3 months
issue commenttensorflow/graphics
Meeting recording is private :(
comment created time in 3 months
pull request commenttensorflow/graphics
@julienvalentin once #310 is through I'll ensure tests are passing, then should be good for review.
@ataiya should I make a separate utils.py file for shared dataset/learning rate functionality shared between fit and train? They're currently in helpers.py, but they're a little project specific...
comment created time in 3 months
pull request commenttensorflow/graphics
@ataiya am I right to go ahead and merge this now? The button has gone green, but I don't want to break anything right off the bat... :)
comment created time in 3 months
Pull request review commenttensorflow/graphics
optimized/debugged data pipeline
# limitations under the License. """Training loop for PointNet v1 on modelnet40.""" # pylint: disable=missing-function-docstring-+import functools import tensorflow as tf +from tqdm import tqdm from tensorflow_graphics.datasets.modelnet40 import ModelNet40 from tensorflow_graphics.nn.layer.pointnet import PointNetVanillaClassifier as PointNet-from tensorflow_graphics.projects.pointnet import augment+from tensorflow_graphics.projects.pointnet import augment as augment_lib
rename / split done. Implementation does NOT rotate the test data.
comment created time in 3 months
pull request commenttensorflow/graphics
optimized/debugged data pipeline
@julienvalentin sorry, got distracted and never got around to addressing comments. I've just made a small tweak as requested by @ataiya (augment -> (augment_rotation, augment_jitter)). I see I can now merge - fairly sure that wasn't an option before. Is that because I accepted contributer invite, or has this been reviewed?
If it's the former and there's still a review process to be had, #312 is a larger PR that encompasses these changes, among other things. The tests there are currently failing just because I'm waiting on #310 . atayia has already gone over much of that. If the overhead to starting a review is high I'd suggest starting there and dropping this one, but if you'd rather smaller PRs then this one's ready to go, and I'll manage the merge conflicts with 312 later.
comment created time in 3 months
push eventjackd/graphics
commit sha b438f322a84d7d2bda4da077ef93a8f1691093d1
tweaked augmentation / preprocessing
push time in 3 months