profile
viewpoint

tensorflow/examples 2555

TensorFlow examples

jdduke/three_cpp 339

A port of three.js to C++

google/XNNPACK 328

High-efficiency floating-point neural network inference operators for mobile, server, and Web

dart-lang/tflite_native 71

A Dart interface to TensorFlow Lite (tflite) through dart:ffi

jdduke/tthread 29

TinyThread++ fork with lambda, future and continuation support.

jdduke/beatdetection 20

Simple beat detection header library

jdduke/fpcpp 19

Functional programming with C++11, inspired by Haskell.

jdduke/glew 3

Tracking GLEW with CMake support

jdduke/Cinder 1

Cinder is a community-developed, free and open source library for professional-quality creative coding in C++.

jdduke/dotfiles 1

.files, including ~/.osx — sensible hacker defaults for OS X

issue commenttensorflow/tensorflow

UnsatisfiedLinkError on nightly build of tensorflow-lite

Also note that the GPU and base TFLite dependencies must have matching versions, i.e.,

    implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly'
    implementation 'org.tensorflow:tensorflow-lite-gpu:0.0.0-nightly'

or

    implementation 'org.tensorflow:tensorflow-lite:2.1.0'
    implementation 'org.tensorflow:tensorflow-lite-gpu:2.1.0'
landicefu

comment created time in 2 days

issue commenttensorflow/tensorflow

UnsatisfiedLinkError on nightly build of tensorflow-lite

Odd, I haven't been able to repro. Can you try clearing your gradle cache and rebulding? Also, what device are you using to test?

I successfully built and deployed using:

cd examples/lite/examples/image_classification/android
./gradlew cleanBuildCachce
./gradlew build
adb install -r -d  ./app/build/outputs/apk/debug/app-debug.apk
landicefu

comment created time in 2 days

issue commenttensorflow/tensorflow

TFLite C/C++ library header file installation

Thanks for flagging this issue. We've been working on something like this, but still need to remove the flatbuffer inclusion from our public C++ APIs. We'll try to prioritize this for early Q2.

thkim-cochl

comment created time in 3 days

pull request commenttensorflow/tensorflow

Additional testcases for embedding_lookup ops

Is this still desired? Is it worth rebasing?

joyalbin

comment created time in 4 days

pull request commenttensorflow/models

Fix: added missing input argument to export_tflite_ssd_graph.py

Is this still necessary?

akhtyamovpavel

comment created time in 4 days

issue commenttensorflow/tensorflow

Build on MacOS for Select Operators TFLite error

Over to @yyoon to help.

rangsaritv

comment created time in 7 days

pull request commenttensorflow/tensorflow

Lite: Space_to_batch Op N-dim support

Looping in @thaink who is helping us improve our kernel ecosystem to support 5D shapes more generally, particularly for operators currently restricted to 4D.

ANSHUMAN87

comment created time in 7 days

pull request commenttensorflow/tensorflow

Lite: Slice Op 4-D bottleneck removed

Hey @ANSHUMAN87 , apologies again for the delayed response. We're working on a more comprehensive proposal for updating our ops to support 5D shapes in particular. Those appear to be the most common scenarios that aren't yet supported, and we'd rather address those first than handle the more general case (which may degrade performance). I'll let @thaink chime in on specific thoughts here, but I think we can extend our 4D support to 5D here without sacrificing performance.

ANSHUMAN87

comment created time in 7 days

PR closed tensorflow/tensorflow

Reviewers
[DOCS] Building Tensorflow with Select Ops Update cla: yes comp:lite size:XS stat:awaiting response

The Android AAR tensorflow-lite-with-select-tf-ops target with select TensorFlow ops has been removed. Now tensorflow-lite-select-tf-ops should used. Updating the documentation to match what's in master branch.

+4 -4

6 comments

1 changed file

sdurandeu

pr closed time in 7 days

pull request commenttensorflow/tensorflow

[DOCS] Building Tensorflow with Select Ops Update

Thanks for flagging, but looks like the docs have been updated separate to address this.

sdurandeu

comment created time in 7 days

pull request commenttensorflow/tensorflow

TFLite: add cmake support

Friendly ping on why bazel is insufficient for your needs. Maintaining two separate build systems is often a losing battle, and the TFLite Makefile exists only for legacy reasons.

jsee23

comment created time in 7 days

Pull request review commenttensorflow/tensorflow

[TFLite int16] 16-bit reference kernel for DEPTHWISE_CONV_2D

 TfLiteStatus Prepare(TfLiteContext* context, TfLiteNode* node) {     if (data_type == kTfLiteUInt8 || data_type == kTfLiteInt8) {       TF_LITE_ENSURE_EQ(context, bias->type, kTfLiteInt32);       TF_LITE_ENSURE_EQ(context, bias->params.zero_point, 0);+    } else if (data_type == kTfLiteInt16) {+      TF_LITE_ENSURE_EQ(context, bias->type, kTfLiteInt64);+      TF_LITE_ENSURE_EQ(context, bias->params.zero_point, 0);+      TF_LITE_ENSURE_EQ(context, input->params.zero_point, 0);

Is this input/output requirement only if the bias is present? Is that enforced in the converter?

wwwind

comment created time in 7 days

Pull request review commenttensorflow/tensorflow

[TFLite int16] 16-bit reference kernel for DEPTHWISE_CONV_2D

 TfLiteStatus EvalQuantizedPerChannel(TfLiteContext* context, TfLiteNode* node,   return kTfLiteOk; } +TfLiteStatus EvalQuantizedPerChannel16x8(+    TfLiteContext* context, TfLiteNode* node, TfLiteDepthwiseConvParams* params,+    OpData* data, const TfLiteTensor* input, const TfLiteTensor* filter,+    const TfLiteTensor* bias, TfLiteTensor* output) {+  DepthwiseParams op_params;+  op_params.padding_type = PaddingType::kSame;

@renjie-liu please file an internal bug if you think we ought to be handling other padding types. As noted, this is matching existing behavior.

wwwind

comment created time in 7 days

Pull request review commenttensorflow/tensorflow

Update ops_version.md

 is supported for every node in Delegation code. ``` const int kMinVersion = 1; TfLiteNode* node;-TfLiteRegistration;+TfLiteRegistration registration = nullptr; context->GetNodeAndRegistration(context, node_index, &node, &registration);

I think the right fix would be to make sure the result is kTfLiteOk here, before proceeding?

kuri-leo

comment created time in 9 days

Pull request review commenttensorflow/tensorflow

[TFLite int16] 16-bit support for reference kernels: MAX/MIN and PACK/UNPACK

 int GetBuiltinOperatorVersion(const OpSignature& op_sig) {       }       return 1; +    case BuiltinOperator_MAXIMUM:+    case BuiltinOperator_MINIMUM:+    case BuiltinOperator_PACK:+      if (op_sig.input_types.at(0) == TensorType_INT8) {

I would avoid grouping pack w/ MAX/MIN, OK to have 2 separate case branches.

wwwind

comment created time in 9 days

pull request commenttensorflow/tensorflow

TransposeConv with Bias

Sorry for the delay, there are some internal conflicts that we have to resolve manually, hoping to resolve in the next day or two.

psunn

comment created time in 10 days

startedVGraupera/1on1-questions

started time in 12 days

Pull request review commenttensorflow/tensorflow

[tflite] export SetNumThreads to TFLite Python API

 class Interpreter(object):   def __init__(self,                model_path=None,                model_content=None,-               experimental_delegates=None):+               experimental_delegates=None,+               num_threads=1):

Let's keep the default None, so it remains implementation-dependent, and we skip setting the threads if it's None.

freedomtan

comment created time in 14 days

Pull request review commenttensorflow/tensorflow

[tflite] export SetNumThreads to TFLite Python API

 def __init__(self,         self._interpreter.ModifyGraphWithDelegate(             delegate._get_native_delegate_pointer())  # pylint: disable=protected-access +    self._interpreter.SetNumThreads(num_threads)

Let's move this call before setting the delegates.

freedomtan

comment created time in 14 days

pull request commenttensorflow/tensorflow

TFLite: add cmake support

@impjdi to comment on selective building of GL/CL. Something like this makes sense to me, and I'm generally supportive, however, I think we might want to take a step back and see if we can restructure the code to more seamlessly allow this, without needing to sprinkle in #ifdefs in many different places.

As for CMake support, I'm sympathetic, but I think we'd need a proper engineering discussion before we move ahead with something like that. The only reason we have the Makefiles, currently, is that cross-compilation w/ Bazel can be difficult for certain systems. Ideally, we'd remove the parallel Makefile build system entirely, and rely on Bazel. Is there a reason Bazel doesn't work for your needs?

jsee23

comment created time in 15 days

pull request commenttensorflow/examples

Add iOS text classification example

@lintian06 , @khanhlvg FYI

Thanks for the link, we'll be sure to add it!

khurram18

comment created time in 15 days

pull request commenttensorflow/tensorflow

[TFLite int16] 16-bit support for reference kernels: MAX/MIN and PACK/UNPACK

Right, having the updated op version should help resolve some of the internal test failures we're seeing.

To do this, you can add logic in GetBuiltinOperatorVersion to detect the input/output types, and use an incremented version, update the op version to TFlite version mapping here, and then update the max version where the builtin is registered here.

I realize this process is fairly tedious, we're hoping to remove some of the boilerplate, but for now that's the general pattern.

wwwind

comment created time in 16 days

Pull request review commenttensorflow/tensorflow

[tflite] bump SPLIT op ver from 1 to 3 in NNAPI delegate

 bool NNAPIDelegateKernel::Validate(              "NNAPI only supports constant int32 axis tensor.", &val_ctx);     } break;     case kTfLiteBuiltinSplit: {-      ExpectOpVersion(version, 1, &val_ctx);+      ExpectOpVersion(version, 3, &val_ctx);

Do you need to filter out this case when running on < 1.2 NNAPI?

freedomtan

comment created time in 16 days

Pull request review commenttensorflow/tensorflow

[tflite] export SetNumThreads to TFLite Python API

 def invoke(self):   def reset_all_variables(self):     return self._interpreter.ResetVariableTensors() +  def set_num_threads(self, i):+    """Set number of threads used by TFLite kernels.++    If not set, kernels are running single-threaded. Note that currently,+    only some kernels, such as conv, are multithreaded.++    Args:+      i: number of threads.+    """+    return self._interpreter.SetNumThreads(i)  class InterpreterWithCustomOps(Interpreter):   """Interpreter interface for TensorFlow Lite Models that accepts custom ops.

Python has much better handling of named, selective arguments for constructors (and functions generally), so I don't think we need an Options construct here. I think we can just add as an other named argument.

freedomtan

comment created time in 16 days

Pull request review commenttensorflow/tensorflow

[tflite] export SetNumThreads to TFLite Python API

 def invoke(self):    def reset_all_variables(self):     return self._interpreter.ResetVariableTensors()++  def set_num_threads(self, i):+    """Set number of threads used by TFLite kernels.++    If not set, kernels are running single-threaded. Note that currently,

Let's remove this API since we have the constructor path.

freedomtan

comment created time in 16 days

issue commenttensorflow/tensorflow

Tensorflow lite dll build failed on Windows

From a master checkout, could you try doing:

bazel build -c opt //tensorflow/lite:tensorflowlite

That should produce a proper .dll, though I want to make sure it's properly symbolized. Thanks!

twkx

comment created time in 16 days

pull request commenttensorflow/tensorflow

[TFLite int16] 16-bit support for reference kernels: MAX/MIN and PACK/UNPACK

Many of our delegates key off of the operator version to know whether it's supported. Are we updating the op versioning for these new 16-bit kernels?

wwwind

comment created time in 16 days

Pull request review commenttensorflow/tensorflow

[tflite] enable INT8 for Java binding

 private void throwIfTypeIsIncompatible(Object o) {       return;     }     DataType oType = dataTypeOf(o);+     if (oType != dtype) {+      // INT8 and UINT8 have the same string name, "byte"+      if (oType.toStringName().equals(dtype.toStringName())) {

Thanks! Can you add a test case in DataTypeTest.java to verify this equality? You can basically just copy this exact logic and make sure it holds for int8/uint8.

freedomtan

comment created time in 17 days

issue commenttensorflow/tensorflow

Tensorflow lite NnApiDelegate crashes on Pixel 3A XL

Thanks for the report. Out of curiosity, since it looks like you used the fp16 compression feature, does the original full float model work properly with NNAPI? Or do you have a link to the fp32 model?

helloworld1

comment created time in 17 days

issue commenttensorflow/tensorflow

Tensorflow lite NnApiDelegate crashes on Pixel 3A XL

Thanks for the report, any chance you could attach the model, and/or a stripped version of the model? Feel free to PM me a link directly if there is any sensitivity, using my alias at google dot com.

helloworld1

comment created time in 17 days

Pull request review commenttensorflow/tensorflow

[lite] pass array_names by const ref

 bool MinMaxApproximatelyEqual(const MinMax& minmax1, const MinMax& minmax2) { // If multiple of these arrays have MinMax, then these are required // to agree with each other. bool PropagateMinMaxAmongArrays(Model* model,-                                const std::vector<string> array_names) {+                                const std::vector<string> &array_names) {

No, I simply mean move the & one space to the left from your original change, so it's next to the type, not the arg name.

gaurav1086

comment created time in 17 days

Pull request review commenttensorflow/tensorflow

[lite] pass array_names by const ref

 bool MinMaxApproximatelyEqual(const MinMax& minmax1, const MinMax& minmax2) { // If multiple of these arrays have MinMax, then these are required // to agree with each other. bool PropagateMinMaxAmongArrays(Model* model,-                                const std::vector<string> array_names) {+                                const std::vector<string> &array_names) {

Nit: the & should go by the type rather than the arg name:

const std::vector<string>& array_names

gaurav1086

comment created time in 17 days

issue closedtensorflow/tensorflow

While compiling `benchmark_model_plus_flex`: error: target 'saved_model_portable_proto' not declared in package

  • other issues: I found similar issues https://github.com/tensorflow/tensorflow/issues/30226 - with similar error message

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: na
  • TensorFlow installed from (source or binary): source
  • TensorFlow version: 2.0, 2.1
  • Python version: 3.6.6
  • Installed using virtualenv
  • Bazel version (if compiling from source): 0.26.1
  • GCC/Compiler version (if compiling from source): clang, ndk 21
  • CUDA/cuDNN version: na
  • GPU model and memory: na

I want to compile benchmark_model_plus_flex for tf v2.1 with ndk 21 but it fails.

I also try to compile both benchmark_model and benchmark_model_plus_flex for ndk18 (as it is the last one offically supported) but it fails with error: undefined reference to 'uselocale' so I switched to ndk21 where at least benchmark_model is building.

Command:

bazel build -c opt   --config=android_arm   --cxxopt='--std=c++11'  \
tensorflow/lite/tools/benchmark:benchmark_model_plus_flex

LOG:

WARNING: /home/k.nowicki/code/tensorflow-master/tensorflow/core/kernels/BUILD:6549:12: in srcs attribute of cc_library rule //tensorflow/core/kernels:android_tensorflow_kernels: please do not import '//tenso
rflow/c/kernels:bitcast_op.cc' directly. You should either move the file to this package or depend on an appropriate rule there
WARNING: /home/k.nowicki/code/tensorflow-master/tensorflow/core/BUILD:2024:12: in srcs attribute of cc_library rule //tensorflow/core:android_tensorflow_lib: please do not import '//tensorflow/c/kernels:ops/
bitcast.cc' directly. You should either move the file to this package or depend on an appropriate rule there
ERROR: /home/k.nowicki/code/tensorflow-master/tensorflow/cc/saved_model/BUILD:40:1: no such target '//tensorflow/core:saved_model_portable_proto': target 'saved_model_portable_proto' not declared in package 
'tensorflow/core' defined by /home/k.nowicki/code/tensorflow-master/tensorflow/core/BUILD and referenced by '//tensorflow/cc/saved_model:reader'
ERROR: Analysis of target '//tensorflow/lite/tools/benchmark:benchmark_model_plus_flex' failed; build aborted: Analysis failed
INFO: Elapsed time: 7.415s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (113 packages loaded, 10710 targets configured)

closed time in 17 days

koodzi

issue commenttensorflow/tensorflow

While compiling `benchmark_model_plus_flex`: error: target 'saved_model_portable_proto' not declared in package

Updated the documentation, between --cxxopt='--std=c++14' and --config=monolithic you should be set.

koodzi

comment created time in 17 days

PR closed tensorflow/tensorflow

Reviewers
Add initial minimal ArmNN delegate plugin. awaiting review cla: yes comp:lite size:XL
  • Adds ArmNN delegate interface
  • Allows to specify backend id
  • Supports the following operators: Conv2d, DepthwiseCond2d, Pool2d, Softmax, Squeeze
+2143 -3

4 comments

16 changed files

GeorgeARM

pr closed time in 17 days

pull request commenttensorflow/tensorflow

Add initial minimal ArmNN delegate plugin.

Tentatively closing per offline discussions, we can reopen after further dialog around hosting and distribution.

GeorgeARM

comment created time in 17 days

Pull request review commenttensorflow/tensorflow

[tflite] enable INT8 for Java binding

 private void throwIfTypeIsIncompatible(Object o) {     DataType oType = dataTypeOf(o);      // INT8 and UINT8 have the same string name, "byte"-    if (oType.toStringName() == dtype.toStringName()) {+    if (oType.toStringName().equals(dtype.toStringName())) {

Thanks! And to avoid unnecessary comparisons, I would move this check underneath the oType != dtype conditional body.

freedomtan

comment created time in 18 days

Pull request review commenttensorflow/tensorflow

[tflite] enable INT8 for Java binding

 private void throwIfTypeIsIncompatible(Object o) {       return;     }     DataType oType = dataTypeOf(o);++    // INT8 and UINT8 have the same string name, "byte"+    if (oType.toStringName() == dtype.toStringName()) {

Nit: avoid using == in Java string comparisons. We should also have a unit test for DataType to ensure equality.

freedomtan

comment created time in 18 days

Pull request review commenttensorflow/tensorflow

[tflite] export SetNumThreads to TFLite Python API

 def invoke(self):   def reset_all_variables(self):     return self._interpreter.ResetVariableTensors() +  def set_num_threads(self, i):+    """Set number of threads used by TFLite kernels.++    If not set, kernels are running single-threaded. Note that currently,+    only some kernels, such as conv, are multithreaded.++    Args:+      i: number of threads.+    """+    return self._interpreter.SetNumThreads(i)  class InterpreterWithCustomOps(Interpreter):   """Interpreter interface for TensorFlow Lite Models that accepts custom ops.

What about making the number of threads an argument to the constructor instead? We're trying to move away from making non-trivial properties mutable after construction.

freedomtan

comment created time in 18 days

PR closed tensorflow/examples

Reviewers
TFLite classification example fix permission crash cla: yes ready to pull

TFLite classification example fix permission crash

+1 -2

4 comments

1 changed file

xiaobailong24

pr closed time in 18 days

pull request commenttensorflow/examples

TFLite classification example fix permission crash

Looks like the issue was fixed separately.

xiaobailong24

comment created time in 18 days

pull request commenttensorflow/examples

Add object detection example for TF Lite with USB webcam.

+Tian to help review

JoHyukJun

comment created time in 18 days

pull request commenttensorflow/examples

Add example for TF Lite detection using a custom model

@srjoglekar246 can you help review, thanks!

Kirill94a

comment created time in 18 days

issue commenttensorflow/tensorflow

While compiling `benchmark_model_plus_flex`: error: target 'saved_model_portable_proto' not declared in package

Thanks for flagging, you'll need to add --config=monolithic to your build invocation. I'll update the documentation for the benchmark_model_plus_flex target to make this clear.

koodzi

comment created time in 21 days

issue commenttensorflow/tensorflow

Supporting a TF lite shared library target

@andreydung can you talk more about your static lib needs? Is there a reason the shared library is insufficient (assuming this is for Android)?

martis-chromium

comment created time in 21 days

issue commenttensorflow/tensorflow

No implementation found for long org.tensorflow.lite.NativeInterpreterWrapper.createErrorReporter(int)

Your latest gradle file doesn't conform to the guidance:

dependencies {
    implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly'
    implementation 'org.tensorflow:tensorflow-lite-select-tf-ops:0.0.0-nightly'
}

Have you given that a try?

The tensorflow-lite-with-select-tf-ops variant is deprecated, and was intended to replace the base tensorflow-lite dependency. However, the new tensorflow-lite-select-tf-ops dependency is intended to be used in addition to the tensorflow-lite dependency. I also noticed that there was a bad rebase in the docs for this feature, which i'll be reverting. However, I would give the approach mentioned above a try. Note that you might need to clear your gradle cache.

daihuidai

comment created time in 21 days

pull request commenttensorflow/tensorflow

TFLite UnityPlugin: Add more interpreter functions on Unity Plugin

And Unity2017 will be out of support soon. May I upgrade Unity version to 2018LTS in another PR? https://blogs.unity3d.com/2018/04/09/new-plans-for-unity-releases-introducing-the-tech-and-long-term-support-lts-streams/

Thanks for flagging, let's wait another quarter and then proceed with the upgrade? And thanks again for the contribution! This is a really nice improvement.

asus4

comment created time in 21 days

PR closed tensorflow/examples

Reviewers
Update README.md cla: yes

I added video tutorial link

+3 -0

4 comments

1 changed file

peyanski

pr closed time in 22 days

pull request commenttensorflow/examples

Update README.md

Hi @peyanski, we appreciate the update, and your video looks quite useful! However, in general we cannot link directly to videos that aren't owned by the broader TensorFlow org.

We are working on a more general community contributions portal, and I'm happy to book mark this such that we could showcase your video along with other community efforts. How does that sound?

peyanski

comment created time in 22 days

Pull request review commenttensorflow/examples

Use output's numDetections instead of static NUM_DETECTIONS

 public static Classifier create(      // Show the best detections.     // after scaling them back to the input size.-    final ArrayList<Recognition> recognitions = new ArrayList<>(NUM_DETECTIONS);-    for (int i = 0; i < NUM_DETECTIONS; ++i) {+      +    // You need to use the number of detections from the output and not the NUM_DETECTONS variable declared on top+      // because on some models, they don't always output the same total number of detections+      // For example, your model's NUM_DETECTIONS = 20, but sometimes it only outputs 16 predictions+      // If you don't use the output's numDetections, you'll get nonsensical data+    int numDetectionsOutput = (int) numDetections[0]; // cast from float to integer

Thanks for the fix! Should this be Math.min(NUM_DETECTIONS, (int) numDetections[0]) for safety?

airman00

comment created time in 23 days

issue commenttensorflow/tensorflow

While compiling `benchmark_model_plus_flex`: error: target 'saved_model_portable_proto' not declared in package

Ah right, TF recently switched to using c++14 instead of c++11, can you update your --cxxopt flags accordingly? Some of the newer code (like Hexagon) does make use of C++14. We're still in the process of updating documentation.

koodzi

comment created time in a month

issue commenttensorflow/tensorflow

While compiling `benchmark_model_plus_flex`: error: target 'saved_model_portable_proto' not declared in package

Actually, can you check and see if you can repro the build failure with the latest checkout from master (rather than 2.1)?

koodzi

comment created time in a month

issue commenttensorflow/tensorflow

While compiling `benchmark_model_plus_flex`: error: target 'saved_model_portable_proto' not declared in package

There were some issues with ndk r18, we typically recommend r17c with a target API of 21.

Looks like the dependency on saved_model for flex is a new one, I'll take a look.

koodzi

comment created time in a month

issue commenttensorflow/tensorflow

Error message is unclear when we failed to load tflite model.

@miaout17 is there a reason we're allowing conversion of this model, only to fail at runtime?

w4-hyunseok

comment created time in a month

issue commenttensorflow/tensorflow

TF-Lite example label Image w shared library

Can you be a bit more explicit? Do you mean you'd like for the example to use one of the shared library targets (either the C or C++ one), rather than statically linking everything into the final binary?

peter197321

comment created time in a month

issue commenttensorflow/tensorflow

Softmax activations don't get converted to Softmax TFLite operator if ndim > 2

@ghop2: If you want to submit a PR for updating the Keras softmax behavior, we'd be happy to approve. Thanks again for flagging the issue.

ghop02

comment created time in a month

issue commenttensorflow/tensorflow

Allowing to provide pre-built flatbuffers when building tflite C++

The next release (2.2) will have the stable C API embedded in the prebuilt Android libraries.

We're also exploring hosting prebuilt libraries for non-Android platforms, but it likely won't be until Q2 that we can make that happen.

DoumanAsh

comment created time in a month

pull request commenttensorflow/tensorflow

Lite: Slice Op 4-D bottleneck removed

Thanks for benchmarking!

I appreciate all the work you've done on this PR, but I'm still struggling to see what improvement this is offering. It doesn't appear any faster, and handles the same 1D-4D cases as the current implementation. If you had a compelling model or use-case which required 5D, I would definitely consider landing this, but until then the benefit isn't as clear. Note that we do need a 5D strided_slice implementation, and have encountered a number of models which require that op be extended.

ANSHUMAN87

comment created time in a month

pull request commenttensorflow/tensorflow

ResolveDilatedConv bug fix

Looks like we don't have good test coverage for this transformation yet, so that's not on you. Just confirming that you were able to run the mode in issue #22146 and you validated the output as being correct?

ANSHUMAN87

comment created time in a month

issue commenttensorflow/tensorflow

Android performance: CPU affinity

Hey @arashb, the blog post itself has been in flux, though we've highlighted the work a bit in recent talks, e.g., at TFWorld. We're planning a more comprehensive performance blog post soon.

alexcohn

comment created time in a month

issue commenttensorflow/tensorflow

Allowing to provide pre-built flatbuffers when building tflite C++

Hey @DoumanAsh , apologies for the severely delayed response.

We have migrated our C API out of experimental, and it does not pull in the flatbuffer headers. At the same time, I'm trying to pull out the flatbuffer header dependency from the C++ API, but that's a bit more involved.

I should also note that we started exporting the C API symbols from the native libraries, and including the C headers, in our prebuilt Android libraries in case you'd prefer to use prebuilts.

DoumanAsh

comment created time in a month

Pull request review commenttensorflow/tensorflow

Lite: Slice Op 4-D bottleneck removed

 inline void PadImageStyle(const tflite::PadParams& op_params, }  template <typename T>-inline void Slice(const tflite::SliceParams& op_params,+inline void Slice(tflite::SliceParams& op_params,                   const RuntimeShape& input_shape,                   const RuntimeShape& output_shape,                   SequentialTensorWriter<T>* writer) {   gemmlowp::ScopedProfilingLabel label("Slice");-  const RuntimeShape ext_shape = RuntimeShape::ExtendedShape(4, input_shape);-  // TODO(dkalenichenko): This op only supports 4D tensors or smaller.-  TFLITE_DCHECK_LE(op_params.begin_count, 4);-  TFLITE_DCHECK_LE(op_params.size_count, 4);-  const int begin_count = op_params.begin_count;-  const int size_count = op_params.size_count;-  // We front-pad the begin and size vectors.-  const int start_b = 4 - begin_count > 0 ? 0 : op_params.begin[0];-  const int stop_b = (4 - size_count > 0 || op_params.size[0] == -1)-                         ? ext_shape.Dims(0)-                         : start_b + op_params.size[0];-  const int start_h = begin_count < 3 ? 0 : op_params.begin[begin_count - 3];-  const int stop_h = (size_count < 3 || op_params.size[size_count - 3] == -1)-                         ? ext_shape.Dims(1)-                         : start_h + op_params.size[size_count - 3];-  const int start_w = begin_count < 2 ? 0 : op_params.begin[begin_count - 2];-  const int stop_w = (size_count < 2 || op_params.size[size_count - 2] == -1)-                         ? ext_shape.Dims(2)-                         : start_w + op_params.size[size_count - 2];-  const int start_d = begin_count < 1 ? 0 : op_params.begin[begin_count - 1];-  const int stop_d = (size_count < 1 || op_params.size[size_count - 1] == -1)-                         ? ext_shape.Dims(3)-                         : start_d + op_params.size[size_count - 1];--  for (int in_b = start_b; in_b < stop_b; ++in_b) {-    for (int in_h = start_h; in_h < stop_h; ++in_h) {-      for (int in_w = start_w; in_w < stop_w; ++in_w) {-        const int len = stop_d - start_d;-        if (len > 0)-          writer->WriteN(Offset(ext_shape, in_b, in_h, in_w, start_d), len);-      }-    }-  }+  const int slice_dimensions = input_shape.DimensionsCount();++  // Below values are under condition if the input is 1-D, as it is a special+  // case, where we will not have next axis to fetch the copy length which is+  // input for memcpy operation below, so we are alligning the variables to the+  // required in the case where input is 1-D. NOTE: This is done only to attain+  // code reuse of compute_slice() defined below.+  bool is_input_1d = slice_dimensions == 1;+  int final_axis = is_input_1d ? 0 : slice_dimensions - 2;+  int copy_len =+      is_input_1d+          ? op_params.size[final_axis] - op_params.begin[final_axis]+          : op_params.size[final_axis + 1] - op_params.begin[final_axis + 1];+  int dim_mul = is_input_1d ? 1 : input_shape.Dims(final_axis + 1);+  int start_fin = is_input_1d ? 0 : op_params.begin[final_axis + 1];++  if (is_input_1d) {+    // (This is very important): Set size so that the final loop will run only+    // once to accommodate complete memcpy operation+    op_params.size[final_axis] = op_params.begin[final_axis] + 1;+  }++  // Store as std::function to allow recursion.+  std::function<void(int, int)> compute_slice =+      [&compute_slice, final_axis, copy_len, dim_mul, start_fin, writer,

Any updates?

ANSHUMAN87

comment created time in a month

issue commenttensorflow/tensorflow

concat axis!=-1 and reshape change data layout don't support quant on dsp or NPU

@miaowang14 what is the expected behavior here?

sunzhe09

comment created time in a month

issue commenttensorflow/tensorflow

TFLite Android Model Benchmark Tool -- results not showing up in adb logcat

@multiverse-tf can you take a look?

forresti

comment created time in a month

pull request commenttensorflow/tensorflow

TransposeConv with Bias

@suharshs can you validate the quantization-specific changes? Thanks.

psunn

comment created time in a month

Pull request review commenttensorflow/tensorflow

TransposeConv with Bias

 int GetOutputDepthFromWeights(const Model& model, const Operator& op) {   return 0; } +bool CheckOpInputSize(const Operator& op) {+  if (op.type == OperatorType::kConv ||+      op.type == OperatorType::kFullyConnected ||+      op.type == OperatorType::kDepthwiseConv) {+        return (op.inputs.size() >= 3);

Can you run the C++ formatter on this PR? https://www.tensorflow.org/community/contribute/code_style#c_coding_style

psunn

comment created time in a month

Pull request review commenttensorflow/tensorflow

TransposeConv with Bias

 namespace toco {  namespace { +int GetBiasIndex(const Operator& op) {+  if (op.type == OperatorType::kConv ||+      op.type == OperatorType::kFullyConnected ||+      op.type == OperatorType::kDepthwiseConv) {+        return 2;+  } else if (op.type == OperatorType::kTransposeConv) {+        return 3;

Same here, reduce indent

psunn

comment created time in a month

Pull request review commenttensorflow/tensorflow

TransposeConv with Bias

 namespace toco {  namespace { +int GetBiasIndex(const Operator& op) {+  if (op.type == OperatorType::kConv ||+      op.type == OperatorType::kFullyConnected ||+      op.type == OperatorType::kDepthwiseConv) {+        return 2;

Nit: redunce indent

psunn

comment created time in a month

PR closed tensorflow/examples

Reviewers
Add iOS text classification example cla: yes
+11685 -0

4 comments

20 changed files

khurram18

pr closed time in a month

pull request commenttensorflow/examples

Add iOS text classification example

Hi @khurram18, just provide a link to your example GitHub repo and we'll try to include it when we roll out a community examples listing. Thanks!

khurram18

comment created time in a month

pull request commenttensorflow/tensorflow

TransposeConv with Bias

I'll be OOO for a few days, will review upon my return early next week (or I'll find somebody else to help review). Thanks again for your patience.

psunn

comment created time in a month

issue commenttensorflow/tensorflow

Supporting a TF lite shared library target

How did you build the 64-bit library? With --config=android_arm64?

martis-chromium

comment created time in a month

issue commenttensorflow/tensorflow

Need tf.signal.rfft op in TFLite

This is great discussion, thanks all for the feedback.

As there are a number of related ops to FFT (RFFT(2D/3D), IRFFT(2D/3D), FFT(2D/3D), ComplexAbs), it would be good to know precisely which op variants are required for your models so we can prioritize support accordingly. If you can link to specific source models, that would be extremely helpful. Thanks!

jpangburn

comment created time in a month

issue commenttensorflow/tensorflow

Failed to load model from file:///android_asset/frozen_inference_graph.pb'

@srjoglekar246 can help in offering guidance for detection model conversion.

SirPhemmiey

comment created time in a month

issue commenttensorflow/tensorflow

Need tf.signal.rfft op in TFLite

We've implemented rfft2d as a custom op which can be used optionally, we could probably do the same for rfft. I've filed an internal request to explore implementing this as a proper builtin op. Thanks for your patience.

jpangburn

comment created time in a month

pull request commenttensorflow/examples

Add iOS text classification example

Hi @khurram18 , can you provide a link to a screenshot of the example?

In general, we typically don't accept wholesale example contributions directly. However, we are working on a way to showcase community-developed and owned examples. Would you be willing to host this example yourself, and we can link to it as part of this community showcase?

khurram18

comment created time in a month

issue commenttensorflow/tensorflow

Hexagon Delegate requirements needs to be clarified?

Thanks for the report. Can you provide more details on the specific make/model of your phone, e.g., adb shell getprop ro.product.device? We've tested execution on 820, so this could be device-specific.

DoumanAsh

comment created time in a month

pull request commenttensorflow/tensorflow

TransposeConv with Bias

Good catch, we should add {{OperatorType::kTransposeConv, 2}, kPendingReleaseOpVersion} as well, thanks!

psunn

comment created time in 2 months

issue commenttensorflow/tensorflow

Failed to load model from file:///android_asset/frozen_inference_graph.pb'

Sounds good, let us know if you run into any issues!

SirPhemmiey

comment created time in 2 months

issue closedtensorflow/tensorflow

Failed to load model from file:///android_asset/frozen_inference_graph.pb'

Hi guys,

I am trying to run the trained model on my android device, but it's crashing on opening.

When i checked my logcat, i have the following screenshot: Screen Shot 2020-01-09 at 5 47 51 PM Process: org.tensorflow.demo, PID: 3797 java.lang.RuntimeException: Failed to load model from 'file:///android_asset/frozen_inference_graph.pb'.

I have tried so many things like changing my TF version to match the tensorflow android build version but all to no avail.

And also, sometimes the logcat will log an error message relating to something like Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.

I would have matched my android build.gradle dependencies with my current TF version but the latest tensorflow-android is 1.13.1.

My model was trained on TF 1.15.0 macOS Mojave 10.14 TF Version: 1.15.0 Model used: ssd_mobilenet_v2_quantized_300*300_coco and ssd_inception_v2_coco

Please help, I have used more than 24 hours in looking for the solution.

closed time in 2 months

SirPhemmiey

issue commenttensorflow/tensorflow

Failed to load model from file:///android_asset/frozen_inference_graph.pb'

TF Mobile is deprecated, so if you need updated support you would have to build the libraries manually from source.

Alternatively, TensorFlow Lite supports both ssd_mobilenet_v2 and ssd_inception, and should offer much better performance and a much smaller binary size. Is there a reason you cannot use TensorFlow Lite?

SirPhemmiey

comment created time in 2 months

Pull request review commenttensorflow/tensorflow

TransposeConv with Bias

 class BaseTransposeConvOpModel : public SingleOpModel {     }     input_ = AddInput(input); +    int bias_size = GetShape(filter_)[0];+    if (input.type == TensorType_FLOAT32) {+      bias_ = AddInput({TensorType_FLOAT32, {bias_size}});

We need to keep tests for the case that don't have a bias. Ideally we would keep the existing tests as is and just add a few additional ones that have biases.

psunn

comment created time in 2 months

Pull request review commenttensorflow/tensorflow

TransposeConv with Bias

 TfLiteStatus Prepare(TfLiteContext* context, TfLiteNode* node) {   TF_LITE_ENSURE(context, input->type == kTfLiteFloat32 ||                               input->type == kTfLiteUInt8 ||                               input->type == kTfLiteInt8);-  TF_LITE_ENSURE_EQ(context, weights->type, input->type);++  if (hasBias) {

has_bias

psunn

comment created time in 2 months

issue commenttensorflow/tensorflow

TFLite No implementation found for long org.tensorflow.lite.NativeInterpreterWrapper.createErrorReporter(int)

The 2.1 release is now available (org.tensorflow:tensorflow-lite:2.1.0), please give it a try.

wosiu

comment created time in 2 months

pull request commenttensorflow/tensorflow

symmetric 16-bit activation quantization

Again, thanks for your patience, the holiday break certainly hasn't helped move things along.

Suharsh will send out some notes/follow-up questions about the API and performance, but otherwise I think we're good to go ahead with a more detailed review.

wwwind

comment created time in 2 months

Pull request review commenttensorflow/tensorflow

Lite: Conv operator optimized

 void TransposeFloatTensor(TfLiteTensor* input, TfLiteTensor* output) {   } } +// Check if im2col needs to be allocated, as some version of optimized Conv dont+// use it. If any change is supporting im2col in any of the Conv versions, then+// it should be updated here as well+bool IsIm2ColRequired(TfLiteTensor* input, TfLiteConvParams* params,+                      TfLiteTensor* filter, OpData* data, bool is_hybrid,+                      KernelType kernel_type) {+  // If HWCN weights are required, Im2Col not required+  if (data->need_hwcn_weights) return false;++  // segregate based on dilated conv & non-dialated conv+  const bool need_dilated_im2col =+      params->dilation_width_factor != 1 || params->dilation_height_factor != 1;+  const bool need_non_dilated_im2col =+      params->stride_width != 1 || params->stride_height != 1 ||+      filter->dims->data[2] != 1 || filter->dims->data[1] != 1;++  const bool need_im2col = need_dilated_im2col || need_non_dilated_im2col;++  // Return early as basic requirement is not met+  if (!need_im2col) return false;++  // Special case for Hybrid, as it supports only non-dilated im2col currently+  const bool is_hybrid_non_dilated = is_hybrid && need_non_dilated_im2col;+  const bool is_quantized =+      input->type == kTfLiteUInt8 || input->type == kTfLiteInt8;++  switch (kernel_type) {+    case kReference:+      if (is_hybrid) {+        return true;+      } else {+        return false;+      }+    case kGenericOptimized:+    case kCblasOptimized:+      if (is_hybrid && !need_non_dilated_im2col) {+        return false;+      } else {+        return true;+      }+    case kMultithreadOptimized:+      if (is_hybrid_non_dilated || is_quantized ||+          !data->supports_multithreaded_kernel) {+        return true;+      } else {+        return false;+      }+    default:+      return false;+  }+}+ // Allocate temporary tensors (`im2col`, `hwcn_weights` if necessary). // Note: `context->AddTensors` might invalidate pointers to existing tensors. // Therefore the logic to add tensors are isolated into this function. static TfLiteStatus AllocateTemporaryTensorsIfRequired(TfLiteContext* context,                                                        TfLiteNode* node,                                                        bool is_hybrid,-                                                       bool is_per_channel) {+                                                       bool is_per_channel,+                                                       KernelType kernel_type) {   auto* params = reinterpret_cast<TfLiteConvParams*>(node->builtin_data);   OpData* data = reinterpret_cast<OpData*>(node->user_data); 

You can remove the filter_width/filter_height vars on lines 213, looks like those are currently breaking the build.

ANSHUMAN87

comment created time in 2 months

issue commenttensorflow/tensorflow

TFLite not support Dynamic input size

We're still actively working on this, and it remains a priority for Q1.

WenguoLi

comment created time in 2 months

issue commenttensorflow/tensorflow

Cannot build gl_delegate of android

Expect a much more detailed and useful guide by the end of January. Thanks for your patience on this.

stereomatchingkiss

comment created time in 2 months

pull request commenttensorflow/examples

Fixed bug

Sorry for the delay, I wasn't cc'ed on the internal review. Thanks for your patience (things move a bit more slowly around the holidays).

kyscg

comment created time in 2 months

issue commenttensorflow/tensorflow

Error: Not in GZIP format

Possibly a Windows-specific issue. @lintian06 could you take a look?

rajputashu

comment created time in 2 months

issue commenttensorflow/tensorflow

Android TFlite inconsistent performance when app is not in focus.

This has resolved it, thank you for the tip!

For completeness, how was it resolved? Using the --define=tflite_with_ruy=false flag? If so, would you mind sharing more about your model so we can investigate ruy-based performance? Thanks!

Artaches

comment created time in 2 months

issue commenttensorflow/models

Depth Multiplier becomes 0, on conversion to TFLITE

I think if you change rate= to dilations= @ https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/testing/op_tests/depthwiseconv.py#L99, it should compile if you run:

bazel test //tensorflow/lite/testing:zip_test_depthwiseconv

Looks like there have been some recent changes that have change the arguments for this op.

SanthoshRajendiran

comment created time in 2 months

issue commenttensorflow/models

Depth Multiplier becomes 0, on conversion to TFLITE

Hmm, good catch, I think there might be a build issue with that specific example test op. Let me take a look.

SanthoshRajendiran

comment created time in 2 months

pull request commenttensorflow/tensorflow

Lite: Space_to_batch Op N-dim support

However if you think limiting the kernel only to 4-Dim is best, i will check and confirm whether the existing logic can be reused.

Worth checking, particularly if this will handle the Conv1D-related scenarios.

ANSHUMAN87

comment created time in 2 months

pull request commenttensorflow/tensorflow

[tflite] fix handling of avgpool in NNAPI delegate

I'm actually still seeing a failure when running on API 29 for this case: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/pooling_test.cc#L559. So I don't think we can land this.

tensorflow/lite/kernels/pooling_test.cc:572: Failure
Value of: m.GetOutput()
Expected: has 1 element that is equal to '\xFF' (255)
  Actual: { '\x1C' (28) } (of type NSt6__ndk16vectorIhNS_9allocatorIhEEEE), whose element #0 doesn't match
freedomtan

comment created time in 2 months

issue commenttensorflow/tensorflow

tensorflow lite build issue on Windows

@terryheo we might look to the core TensorFlow libs for guidance here, see also https://github.com/tensorflow/tensorflow/blob/master/tensorflow/BUILD#L728

TheArheus

comment created time in 2 months

issue commenttensorflow/tensorflow

tensorflow lite build issue on Windows

Just to confirm, are y'all using the latest master checkout? or the 2.0 branch?

TheArheus

comment created time in 2 months

issue commenttensorflow/tensorflow

TFLite No implementation found for long org.tensorflow.lite.NativeInterpreterWrapper.createErrorReporter(int)

@alexeyvasilyev the fix did not make it into 2.0, but it will be in the upcoming 2.1 release (expected to be finalized soon). In the meantime, please try the nightly build.

wosiu

comment created time in 2 months

Pull request review commenttensorflow/tensorflow

TFLite UnityPlugin: Add more interpreter functions on Unity Plugin

 namespace TensorFlowLite   /// </summary>   public class Interpreter : IDisposable   {-    private const string TensorFlowLibrary = "tensorflowlite_c";+    public struct Options: IEquatable<Options> {+      /// <summary>+      /// The number of CPU threads to use for the interpreter.+      /// </summary>+      public int threads;++      public bool Equals(Options other) {+        return threads == other.threads;+      }+    }++    public struct TensorInfo {+      public string name { get; internal set; }+      public DataType type { get; internal set; }+      public int[] dimensions { get; internal set; }+      public QuantizationParams quantizationParams { get; internal set; }++      public override string ToString() {+        return string.Format("name: {0}, type: {1}, dimensions: {2}, quantizationParams: {3}",+          name,+          type,+          "[" + string.Join(",", dimensions) + "]",

e.g., string.Join(",", dimensions.Select(d => d.ToString()).ToArray());

asus4

comment created time in 2 months

more