profile
viewpoint
Brennan Saeta saeta Google Tech lead & manager for the Swift for TensorFlow (S4TF) team in Google Brain.

apple/swift 50685

The Swift Programming Language

fastai/fastai_dev 512

fast.ai early development experiments

fastai/swiftai 319

Swift for TensorFlow's high-level API, modeled after fastai

google-research/swift-tfp 123

Find shape errors before you run your code!

saeta/play-deb-packaging 25

A souce snapshot of a sample Play! app with SBT packaging.

saeta/penguin 20

Data tables in Swift

saeta/CS-249A-Reimplementation-In-Scala 10

I am simply reimplementing an 8000 line C++ Project for CS 249A @ Stanford in Scala to learn about scala, and to experiment with ideas in software engineering.

saeta/axess 3

A tool to automatically scan websites for simple accessibility issues.

push eventapple/swift

zoecarver

commit sha d9b95a026f115c2981a80dee547e8cd4d871d3d9

Implement devirtualization calls to composition type protocols

view details

zoecarver

commit sha bfb3dd72d779a13afd49a28b365c66e1b90f49af

Clean up implementation and remove debug

view details

zoecarver

commit sha 287bbd4df40cdf96549894b4c378a55ac9c5e26e

Add a test

view details

zoecarver

commit sha f3ac3bd08b6bbece20fd20567963590ef63b5432

Cleanup and fix spacing (again)

view details

zoecarver

commit sha f1c75f6d628fa47996cf1bafba7ed51162bd626f

Remove assertion all together and use getAddressOfStackInit instead of dyn_cast to StoreInst

view details

zoecarver

commit sha 4b36f3fd69d10fcc2e1ba405847f0da8747359c6

Fix based on review and add more tests

view details

zoecarver

commit sha f1c15d26f30ca5abfffe633daf0f97ab0bda2d8c

Fix tests

view details

zoecarver

commit sha 6c48a8d35fd5a821408ce29475db38b1fe72267a

Ensure that only correct patterns are optimized

view details

zoecarver

commit sha 01a452f0cb72a42a84f2af25c92e9d4cc0b4130c

Update comment

view details

zoecarver

commit sha 17894bb6eb8f85161bd3d6369693b3718ca70035

Remove redundant protocols

view details

zoecarver

commit sha febbb702e483f1d7638dc212cb64cac19051c51d

Merge branch 'master' into fix/substitution-map-composition

view details

zoecarver

commit sha 3ab158ef687f009a21d596547b6dd6b13702236f

Merge branch 'master' into fix/substitution-map-composition

view details

zoecarver

commit sha 2913dbc430579b9fba575204473f660d847b2604

Check dominance order and add .sil test

view details

zoecarver

commit sha 331a6af2a422e376a733f3800e06b9f355dfeeec

Remove .dump in unrelated file

view details

zoecarver

commit sha d032de63a7db6d558d85914b5e6a2b2fd0d290d6

Move relevant part of sil test to the top of the file

view details

zoecarver

commit sha 644689dc12082942800327074c76d97851b07af8

Rename test

view details

Jonathan Keller

commit sha e1ceb4f437471ca836f89dde72b3329b468d5f70

[SILOptimizer] Generalize optimization of static keypaths We have an optimization in SILCombiner that "inlines" the use of compile-time constant key paths by performing the property access directly instead of calling a runtime function (leading to huge performance gains e.g. for heavy use of @dynamicMemberLookup). However, this optimization previously only supported key paths which solely access stored properties, so computed properties, optional chaining, etc. still had to call a runtime function. This commit generalizes the optimization to support all types of key paths.

view details

zoecarver

commit sha 6fcf437f1c0a0f7b8e30867b943891be1d95960a

Merge branch 'master' into fix/substitution-map-composition

view details

zoecarver

commit sha 758f1f3208ca897ec2a8f53b93c8e323feb0b855

Update getAddressOfStackInit and getStackInitInst to support store instructions

view details

zoecarver

commit sha f4e3ca9999a422b2c2d907778e6cb56c4ee63833

Formatting

view details

push time in 2 days

PR merged apple/swift

Tensorflow merge tensorflow

Merge master into the S4TF tensorflow branch.

+26875 -12919

9 comments

727 changed files

saeta

pr closed time in 2 days

Pull request review commenttensorflow/swift-models

Transformer: replace Python with Swift objects

 extension BytePairEncoder {     }      internal static let defaultGlossary: [String] = [-        "e.g", "i.e", "&", "|", "<", ">", "'", """, "[", "]"]+        "e.g", "i.e", "&", "|", "<", ">", "'", """, "[", "]",+    ]      internal static let defaultGlossaryRegex: NSRegularExpression = {         let escapedGlossary = defaultGlossary.map { "\\Q\($0)\\E" }.joined(separator: "|")         return try! NSRegularExpression(pattern: "(?:\(escapedGlossary))|(?!\(escapedGlossary))")     }() +    /// Regular expression matching the OpenAI GPT-2 implementation.+    internal static let gpt2Glossary: [String] = [
    internal static let gpt2Glossary = [

Does the type not get inferred automatically?

texasmichelle

comment created time in 2 days

Pull request review commenttensorflow/swift-models

Transformer: replace Python with Swift objects

 public struct BytePairEncoder {          // Check if the new words parts are in the vocabulary, and backtrack if necessary.         let encoded = parts.flatMap { part -> [String] in-        if vocabulary.contains(part) { return [part] }+            if vocabulary.contains(part) { return [part] }             return splitRecursively(part)         }          // Update the cache and return.         // if useCache { cache[token] = encoded }         return encoded     }++    /// Encodes the provided token to a sequence of BPE-coded tokens.+    ///+    /// - Parameters:+    ///   - token: Token to encode.+    /// - Returns: Array containing the BPE-coded tokens.+    public func encodeGpt2(token: String) -> [String] {+        // Split into parts before encoding.+        var parts = BytePairEncoder.splitWithDelimitersGpt2(+            token: token,+            glossaryRegex: BytePairEncoder.gpt2GlossaryRegex)+        if parts.count < 2 {+            // Encode the full token and return.+            return parts.map({ BytePairEncoder.encodedToken($0) })
            return parts.map { BytePairEncoder.encodedToken($0) }

Does the suggestion compile? If so, is that preferable?

texasmichelle

comment created time in 2 days

Pull request review commenttensorflow/swift-models

Transformer: replace Python with Swift objects

 extension BytePairEncoder {     }      internal static let defaultGlossary: [String] = [-        "e.g", "i.e", "&amp;", "&#124;", "&lt;", "&gt;", "&apos;", "&quot;", "&#91;", "&#93;"]+        "e.g", "i.e", "&amp;", "&#124;", "&lt;", "&gt;", "&apos;", "&quot;", "&#91;", "&#93;",+    ]      internal static let defaultGlossaryRegex: NSRegularExpression = {         let escapedGlossary = defaultGlossary.map { "\\Q\($0)\\E" }.joined(separator: "|")         return try! NSRegularExpression(pattern: "(?:\(escapedGlossary))|(?!\(escapedGlossary))")     }() +    /// Regular expression matching the OpenAI GPT-2 implementation.+    internal static let gpt2Glossary: [String] = [+        "'s", "'t", "'re", "'ve", "'m", "'ll", "'d", " ?\\p{L}+", " ?\\p{N}+",+        " ?[^\\s\\p{L}\\p{N}]+", "\\s+(?!\\S)", "\\s+",+    ]++    internal static let gpt2GlossaryRegex: NSRegularExpression = {+        let escapedGlossary = gpt2Glossary.map { $0 }.joined(separator: "|")
        let escapedGlossary = gpt2Glossary.joined(separator: "|")

Do you mean this?

texasmichelle

comment created time in 2 days

Pull request review commentapple/swift

[build] Fix TensorFlow configure script invocation.

 def should_build(self, host_target):         return self.args.build_tensorflow_swift_apis      def build(self, host_target):-        with shell.pushd(self.source_dir):-            shell.call([-                os.path.join(self.source_dir, '..', 'swift', 'utils',-                             'configure-tensorflow-defaults'),-            ])-            shell.call([-                self.toolchain.bazel,-                "build",-                "-c", "opt",-                "--define", "framework_shared_object=false",-                "//tensorflow:tensorflow",-            ])+        # Run the TensorFlow configure script: `yes "" | ./configure`.+        configure_script_path = os.path.join(self.source_dir, 'configure')+        yes_process = subprocess.Popen(['yes', ''], stdout=subprocess.PIPE)

Note: I think that it's possible that upstream Apple folks won't like this, as they would like all filesystem interactions to go through their shell wrappers. (Or so I've been told.)

dan-zheng

comment created time in 2 days

Pull request review commenttensorflow/swift-models

Initial implementation of a TensorFlow v2 checkpoint writer

+// Copyright 2020 The TensorFlow Authors. All Rights Reserved.+//+// Licensed under the Apache License, Version 2.0 (the "License");+// you may not use this file except in compliance with the License.+// You may obtain a copy of the License at+//+//     http://www.apache.org/licenses/LICENSE-2.0+//+// Unless required by applicable law or agreed to in writing, software+// distributed under the License is distributed on an "AS IS" BASIS,+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+// See the License for the specific language governing permissions and+// limitations under the License.++import Foundation+import TensorFlow++/// A Swift-native TensorFlow v2 checkpoint writer. This writer has no dependencies+/// on the TensorFlow runtime or libraries.+open class CheckpointWriter {+    // TODO: Extend handling to different tensor types.+    let tensors: [String: Tensor<Float>]++    /// Initializes the checkpoint reader from a dictionary of tensors, keyed on their string names.+    ///+    /// - Parameters:+    ///   - tensors: A dictionary containing the tensors to be written, with the keys being the+    ///     names of those tensors to write in the checkpoint.+    public init(tensors: [String: Tensor<Float>]) {+        self.tensors = tensors+    }++    /// Writes the checkpoint to disk, in a specified directory. A TensorFlow v2 checkpoint consists+    /// of a directory that contains a [name].index header file and one or more+    /// [name].data-0000X-of-0000Y binary shard files with the tensor bytes within them.+    ///+    /// - Parameters:+    ///   - directory: The directory to write the checkpoint into. If it doesn't exist, it will be+    ///     created.+    ///   - name: The base name of the checkpoint, which is what will have the .index and+    ///     .data-0000X-of-0000Y extensions appended to it for files in the checkpoint directory.+    public func write(to directory: URL, name: String) throws {+        try createDirectoryIfMissing(at: directory.path)+        let indexWriter = CheckpointIndexWriter(tensors: tensors)+        let indexHeader = indexWriter.serializedHeader()+        let headerLocation = directory.appendingPathComponent("\(name).index")+        try indexHeader.write(to: headerLocation)++        // TODO: Handle splitting into multiple shards.+        try writeShard(+            to: directory.appendingPathComponent("\(name)"), shard: 0, numShards: 1,+            tensorList: indexWriter.orderedTensors)+    }++    func writeShard(to location: URL, shard: Int, numShards: Int, tensorList: [String]) throws {+        let shardFile = CheckpointReader.shardFile(+            location: location, shard: shard, totalShards: numShards)++        var outputBuffer = Data()+        // TODO: Write this directly to disk, rather than accumulating it in memory.+        for tensorName in tensorList {+            guard let tensor = tensors[tensorName] else {+                fatalError("Mismatch in sorted tensors at name: \(tensorName).")+            }+            let scalars = tensor.array.scalars+            // scalars.withUnsafeBytes { pointer in

Tinkercruft?

BradLarson

comment created time in 2 days

Pull request review commenttensorflow/swift-models

Initial implementation of a TensorFlow v2 checkpoint writer

 for _ in 0..<100 {     print(encoder.decode(tokens[0].makeNumpyArray()), terminator: "") } print()++// The following illustrates how to write out a checkpoint from this model and load it back in.+/*+var tensors = [String: Tensor<Float>]()+recursivelyObtainTensors(model, scope: "model", tensors: &tensors, separator: "/")++let writer = CheckpointWriter(tensors: tensors)+try writer.write(to: temporaryDirectory, name: "model2.ckpt")++let reader2 = try CheckpointReader(+    checkpointLocation: temporaryDirectory.appendingPathComponent("model2.ckpt"),+    modelName: "Transformer",+    additionalFiles: [])++let recreatedmodel = TransformerLM(reader: reader2, config: config, scope: "model")+*/

Nit: trailing newline?

BradLarson

comment created time in 2 days

push eventtensorflow/swift-models

Saleem Abdulrasool

commit sha 070bca7be9e8a0b7bbc8c9b1724aac97256d657e

Transformer: link against `TextModels` (#339) The transformer model uses `TextModels` but did not link against it, which resulted in dependency issues. This fixes the build on Windows.

view details

push time in 2 days

PR merged tensorflow/swift-models

Reviewers
Transformer: link against `TextModels`

The transformer model uses TextModels but did not link against it, which resulted in dependency issues. This fixes the build on Windows.

+1 -0

0 comment

1 changed file

compnerd

pr closed time in 2 days

push eventtensorflow/swift-apis

t.ae

commit sha 84e0da0be6acb56cc45e3991e59061ad5cadd130

Refine tiled (#689) Add tiled API and tests with refined preconditions.

view details

push time in 2 days

PR merged tensorflow/swift-apis

Refine tiled

This PR Adds

  • New API tiled(multiples: [Int])
  • Precondition for tiled
  • Tests for tiled
+40 -0

0 comment

3 changed files

t-ae

pr closed time in 2 days

pull request commenttensorflow/swift-apis

Improve compilation time of tests

Thank you @t-ae !!

t-ae

comment created time in 2 days

Pull request review commenttensorflow/swift-apis

Refine tiled

 public extension Tensor {     ///     /// - Precondition: The expected `rank` of multiples must be `1`.     /// - Precondition: The shape of `multiples` must be `[tensor.rank]`.+    /// - Precondition: All scalars in `multiples` must be non-negative.     @inlinable     @differentiable(wrt: self where Scalar: TensorFlowFloatingPoint)     func tiled(multiples: Tensor<Int32>) -> Tensor {         precondition(multiples.rank == 1, "The expected rank of multiples must be 1.")         precondition(             rank == multiples.shapeTensor.scalarized(),             "The shape of multiples must be [tensor.rank].")+        precondition((multiples .>= 0).all(), "All scalars in multiples must be non-negative.")

At a high level, it's relatively "cheap" to do preconditions against Swift values, and Tensor metadata (e.g. shape, rank), but it's much more expensive (performance wise) to make assertions about the values of Tensors. (This is because Tensors logically represent an asynchronously computed "future" value (i.e. coming from an accelerator) that might not be available yet. Asserting preconditions about the values of tensors forces a device synchronization step, which is expensive for performance.)

You're right that both implementations of tiled should have this precondition, but for performance reasons, could we only have this precondition in the implementation above, and make the assertion on [Int]? (e.g. using multiples.allSatisfy { $0 >= 0 } or something like that?)

Aside: if an element in multiples is equal to 0, doesn't that mean that the resulting tensor contains zero elements? Do we want this as a possibility, or should we have the precondition be $0 > 0? Or am I confused?

t-ae

comment created time in 2 days

Pull request review commenttensorflow/swift-apis

Refine tiled

 public extension Tensor {             splitDim: Tensor<Int32>(Int32(axis)),             numSplit: Int64(sizes.shape[0]))     }+    +    /// Returns a tiled tensor, constructed by tiling this tensor.+    ///+    /// This constructor creates a new tensor by replicating this tensor `multiples` times. The+    /// constructed tensor's `i`'th dimension has `self.shape[i] * multiples[i]` elements, and the+    /// values of this tensor are replicated `multiples[i]` times along the `i`'th dimension. For+    /// example, tiling `[a b c d]` by `[2]` produces `[a b c d a b c d]`.+    ///+    /// - Precondition: The expected `rank` of multiples must be `1`.+    /// - Precondition: The shape of `multiples` must be `[tensor.rank]`.+    /// - Precondition: All scalars in `multiples` must be non-negative.+    @inlinable+    @differentiable(wrt: self where Scalar: TensorFlowFloatingPoint)+    func tiled(multiples: [Int]) -> Tensor {+        // TODO(TF-433): Remove workaround for differentiating `map`.

Flagging this for @dan-zheng / @marcrasi .

t-ae

comment created time in 2 days

push eventapple/swift

Jonathan Keller

commit sha e1ceb4f437471ca836f89dde72b3329b468d5f70

[SILOptimizer] Generalize optimization of static keypaths We have an optimization in SILCombiner that "inlines" the use of compile-time constant key paths by performing the property access directly instead of calling a runtime function (leading to huge performance gains e.g. for heavy use of @dynamicMemberLookup). However, this optimization previously only supported key paths which solely access stored properties, so computed properties, optional chaining, etc. still had to call a runtime function. This commit generalizes the optimization to support all types of key paths.

view details

Rintaro Ishizaki

commit sha 4ced714e2508723cbfa6d550a53db6524d810bfb

[CodeCompletion] Allow type variable in MakeAbstractConformanceForGenericType When typesSatisfyConstraint() is called with 'openArchetypes=true', archetypes are substituted with type variables. If they have conformances, they used to hit assertion in 'MakeAbstractConformanceForGenericType::operator()'. Adjust the assetion to accept 'TypeVariableType'. rdar://problem/56834798

view details

Jonathan Keller

commit sha 48132ea22308118cfb175ab300e2d641be36bb73

[SILOptimizer] address key path projection review feedback

view details

Luciano Almeida

commit sha 968a37d2218f749921ea459128dfe345d83211d3

[Diagnostics] Tailored diagnostic message for conditional downcast involving literals that can be statically coerced

view details

Luciano Almeida

commit sha fd2dbe31353ee6611d65244a1c54b6f6fdee7da7

[tests] Adding literal downcast tests

view details

Jonathan Keller

commit sha d7acf12158b1cff6f5f23da82f8b2c9149c43464

[SILOptimizer] fix crash in key path projector When a computed property returns a generic, the accessor's function type may involve a type parameter that needs to be resolved using the key path instruction's substitution map.

view details

Jonathan Keller

commit sha a06fe96fd9a7d789bde2ca34a1d5b2d71c5571d0

[SILGen] fix key path setter access for @testable emitKeyPathComponentForDecl was only checking if the setter was accessible from the current module, not the current function. This failed when accessing an internal setter from a module imported for testing.

view details

Jonathan Keller

commit sha 1feead804b6b946a1323f7360685b9c64aae3df4

[SILOptimizer] fix KeyPathProjector memory management I was inconsistently providing initialized or uninitialized memory to the callback when projecting a settable address, depending on component type. We should always provide an uninitialized address.

view details

Pavel Yaskevich

commit sha 0a8de8bda845ca2e4a8d29344c42d76468655f85

[ConstraintSystem] Allow `simplifyRestrictedConstraintImpl` to diagnose contextual failures with optionals Since `simplifyRestrictedConstraintImpl` has both parent types and does nested type matching it's a good place to diagnose top-level contextual problems like mismatches in underlying types of optionals.

view details

Pavel Yaskevich

commit sha 802e141be97eb619ba3ac4d7644a5148f2e84fda

[Diagnostics] Diagnose mismatches related to use of `===` and `!==` via a tailored fix

view details

Pavel Yaskevich

commit sha bb656e120d875fbd4ae4bde0fb8a654c0e3d1da4

[CSFix] Teach `getStructuralTypeContext` about tuple initialization calls

view details

Pavel Yaskevich

commit sha f6b7df161c3c42d5ab963b696dc4e5771fc0b5bc

[ConstraintSystem] Increase impact of a missing conformance related to stdlib type Prioritize type mismatches over conformance failures when stdlib types are involved because it wouldn't be appropriate to suggest to add such a conformance, so the problem is most likely related to something else e.g. other overload choice has a better fix. Consider following example: ```swift struct S { init(_: Double) {} init<T: BinaryInteger>(_: T) {} } _ = S(Double("0")) ``` In cases like that it's better to prefer failable initializer which takes a `String` and returns `Double?` and diagnose a problem related to missing optional unwrap instead of missing conformances related to a `String` argument of other `Double` initializer just because it returns a concrete type.

view details

Pavel Yaskevich

commit sha 0c451bb2a9580160a31153856d237c2d5717cd4b

[Diagnostics] Diagnose ambiguities related to ephemeral pointers Diagnose ambiguity related to overloaded declarations where only *some* of the overload choices have ephemeral pointer warnings/errors associated with them. Such situations have be handled specifically because ephemeral fixes do not affect the score. If all of the overloads have ephemeral fixes associated with them it's much easier to diagnose through notes associated with each fix.

view details

Pavel Yaskevich

commit sha 79b757d3cd8ab3e11ff6967ffcddc1c907d5c60e

[ConstraintSystem] Detect and diagnose mismatches in single parameter function conversions When there is a conversion from e.g. `(A) -> Void` to `(B) -> Void` matching between `A` and `B` is going to have a special locator which doesn't end in `TupleElement`, so `repairFailures` has to account for that and fix it just like regular argument mismatch. Resolves: rdar://problem/59066040

view details

Pavel Yaskevich

commit sha df21cbf85ce58119a03cbd1885db4c1802e77f1b

[ConstraintSystem] Ignore attempt to bind type var to dependent member with incorrect base Just like in cases where both sides are dependent member types with resolved base that can't be simplified to a concrete type let's ignore this mismatch and mark affected type variable as a hole because something else has to be fixed already for this to happen.

view details

Pavel Yaskevich

commit sha e17df6bcdf48b274099ef9f0d27f1cca3c1c1e73

[CSGen] Don't try to infer closure types with destructured parameters If one of the parameters represents a destructured tuple e.g. `{ (x: Int, (y: Int, z: Int)) in ... }` let's fail inference and not attempt to solve the constraint system because: a. Destructuring has already been diagnosed by the parser; b. Body of the closure would have error expressions for each incorrect parameter reference and solver wouldn't be able to produce any viable solutions.

view details

Pavel Yaskevich

commit sha ca91a1bee02df591c1015c4268a640f46b896215

[ConstraintSystem] Detect and diagnose OoO arguments in call to binary functions/operators Identify problems like: ```swift func foo(_ x: Int, _ y: String) {} func bar(a: Int, b: String) { foo(b, a) // Instead of `foo(a, b)` } ``` Where arguments are out-of-order and repair it by using OoO fix on the parent locator.

view details

Pavel Yaskevich

commit sha 81addcc8c34688adbd006323d77db41bf044b699

[CSDiag] NFC: Remove `visitApplyExpr` and associated logic

view details

Pavel Yaskevich

commit sha 2565d418c71727ae63583131e2c112d0ecf62a44

[ConstraintSystem] NFC: Remove obsolete `CalleeCandidateInfo`

view details

Pavel Yaskevich

commit sha 8722530943aca3a3b6845b7e54a968c0505e6b73

[CSDiag] NFC: Move `diagnoseDeprecatedConditionalConformanceOuterAccess` to CSApply

view details

push time in 3 days

PR opened apple/swift

Tensorflow merge

Merge master into the S4TF tensorflow branch.

+19981 -8768

0 comment

512 changed files

pr created time in 3 days

push eventapple/swift

Brennan Saeta

commit sha 70fd2e60d78f8e48f8b664949599603ae60e621f

Fix all tests in merge branch.

view details

push time in 3 days

push eventapple/swift

Brennan Saeta

commit sha 8baa0e6002eacc006d6b0bd07bb9c4bd00e06a1d

Switch to `#if canImport(PythonKit)` for tests.

view details

push time in 3 days

push eventapple/swift

Brennan Saeta

commit sha 965e53ee50ed0d88d9c2b7c53c55a5584b956135

Fix build toolchain for TensorFlow. Applies https://github.com/apple/swift/pull/29958 and applies the same logic to tensorflow.py.

view details

push time in 3 days

pull request commentapple/swift

build: improve the PythonKit build

Will test shortly. LGTM.

compnerd

comment created time in 3 days

pull request commenttensorflow/swift-apis

Fix SequentialTests

Right, but given that it's trying to predict values in the set of [0, 1], I was a little suspicious that relu would do that. It also seems like a wacky test, as we're using a bunch of different optimizers simultaneously...

t-ae

comment created time in 3 days

pull request commenttensorflow/swift-apis

Fix SequentialTests

Thanks for the PR! I was initially very confused, as the kokoro checks passed for #678. I took a quick look at the CI log for that PR, and found that SequentialTests weren't run. A bit of sleuthing showed that they were disabled in https://github.com/tensorflow/swift-apis/commit/3304db3e728120b55674cca06894b6ea5083b5e8 and haven't been re-enabled.

When I reenable the sequential tests, I see the following error on a recent toolchain:

/usr/local/google/home/saeta/src/s4tf2/tensorflow-swift-apis/Tests/TensorFlowTests/SequentialTests.swift:65: error: SequentialTests.testSequential : XCTAssertEqual failed: ("[[0.0],
 [0.0],
 [0.0],
 [0.0]]") is not equal to ("[[0.50378805],
 [0.50378805],
 [0.50378805],
 [0.50378805]]") - 

Since I am suspicious of the all-zero result (although the test appears to be a little wacky), it seems like this might be catching a worthwhile error, and so I'd prefer that we don't just convert it to a shape check. Does that make sense?

CC @asuhan / @sgugger / @dan-zheng

t-ae

comment created time in 3 days

pull request commentapple/swift

[TF merge] Fix macOS toolchain builds.

Thanks for this! I just fixed the merge bot, which broke. It should merge from tensorflow -> tensorflow-stage automatically, so things will get picked up into the tensorflow-merge branch on the next merge, IIUC. CC @compnerd to correct me if I'm wrong...

dan-zheng

comment created time in 4 days

push eventtensorflow/swift-apis

t.ae

commit sha 6a67b4ef646222af1dc345d631019cc3602eb16b

Add precondition for BatchNorm, LayerNorm (#677)

view details

push time in 4 days

PR merged tensorflow/swift-apis

Add precondition for BatchNorm, LayerNorm kokoro:run

offset and scale in BatchNorm and LayerNorm are feature-wise parameters. AFAIK their shapes must be [feature count], but there's no restriction for their shapes for now.

If their shapes are wrong, they can cause crash in callAsFunction when they are reshaped. https://github.com/tensorflow/swift-apis/blob/3304db3e728120b55674cca06894b6ea5083b5e8/Sources/TensorFlow/Layers/Normalization.swift#L175-L178

This PR restricts their shapes by 2 steps.

  1. Check if their ranks are 1 and if their shapes are same in initializer
  2. Check if their shapes are [feature count] in callAsFunction

I had to separate them because the feature count is not available in initializer.

Related: #517

+12 -0

0 comment

1 changed file

t-ae

pr closed time in 4 days

push eventtensorflow/swift-models

Saleem Abdulrasool

commit sha d191bc4b342d270b7a45ce4a0390e8b542720149

docs: document CMake support (#306) Add some documentation on how to use the CMake based build and some of the limitations of it.

view details

push time in 4 days

PR merged tensorflow/swift-models

Reviewers
docs: document CMake support kokoro:run

Add some documentation on how to use the CMake based build and some of the limitations of it.

+44 -0

0 comment

1 changed file

compnerd

pr closed time in 4 days

push eventsaeta/penguin

Brennan Saeta

commit sha 8769b59295161d0bcadc3db684b5cb2f11cd5a7d

Add StringSummary information.

view details

Brennan Saeta

commit sha 2ab73be2ad9b39bcc2d4a192204d9d0f15b48a34

Add summaries everywhere!

view details

push time in 4 days

push eventapple/swift

Brennan Saeta

commit sha 5fd9397c29f2136a0471c51a3978a898142ec1f5

PythonKit fixes & misc.

view details

push time in 5 days

startedeaplatanios/jelly-bean-world

started time in 5 days

push eventapple/swift

Brennan Saeta

commit sha a599dda71067bf6d8b22b736cfa2ef56a886c059

More build fixings!

view details

push time in 5 days

push eventapple/swift

Brennan Saeta

commit sha b0b05e69467a9876ceff35e8ec1e7dffca436a0c

WIP: Getting toolchains to build again. compnerd@ to continue. :-)

view details

push time in 5 days

create barnchapple/swift

branch : tensorflow-merge

created branch time in 5 days

create barnchapple/llvm-project

branch : swift/tensorflow-merge

created branch time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha ee75e70bee7d9e28bbf469cb12b45f49661beeda

Add initial summary implementations.

view details

push time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha afc2595a3f1cb927d3fbbb891c7ba8fbab3933bf

Reduce visibility of computed property.

view details

push time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha 4739358e18d17ffc10e7ee88d29212898c1cad94

Fix incorrect reduce, min, and max computations.

view details

push time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha de0d8d9518749935046a1817bf6c5461ac326094

Gardening: remove deprecated subscript on PTypedColumn.

view details

push time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha 6ac2b457ed35836bb8f8d45a06c19f9fc4f5db4a

Add bool conversions & some additional tests.

view details

Brennan Saeta

commit sha d4c9912275bc3e58a449eced94368970981ffabd

Add column type to the display string.

view details

push time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha 45a75e36c3a44e0d4ae30b9029900567de4dac6a

Refactor to disambiguate PTypedColumn and PColumn.

view details

push time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha 0bdba1e52b9c88c05d3e087dd17de57f32be04aa

Remove old deprecated CSV story.

view details

Brennan Saeta

commit sha 17d28e5f19a06e1c7d19f24fb1b7a3c325446279

Fix bug in map implementation that didn't handle missing values correctly.

view details

Brennan Saeta

commit sha 6620739cba02353c1257a234089ad2d07e997481

Fix parsing issues in parsing the criteo TSV file.

view details

push time in 5 days

push eventsaeta/penguin

Brennan Saeta

commit sha d1bfe57c9f79d0bc48458ded36b46f862c4bd18a

Conform Bool to Comparable in order to use Bool's in PTypedColumn's.

view details

Brennan Saeta

commit sha 57561947e4f9c0e972c23dc1860323461ab2933d

Successfully loads the first 2k rows of the Criteo dataset!

view details

Brennan Saeta

commit sha bb7d8eeb57e2a6e6193a77e4277610ee4b16a5a8

Clean up some warnings.

view details

push time in 6 days

push eventsaeta/penguin

Brennan Saeta

commit sha 1f042888ae22ab8cd70ee0a86f8776a31a874830

Add String to PStringParsible.

view details

Brennan Saeta

commit sha 5c89b3de477e9b1816cc2979b4ae8a69b17327a5

Add append functionality to Columns & IndexSet's. This will be important for incremental parsing of CSVs.

view details

push time in 6 days

push eventsaeta/penguin

Brennan Saeta

commit sha fdea6b8b4008a39e51c2dd05a92b0899e0381f8d

Add support for string parsible.

view details

push time in 6 days

push eventsaeta/penguin

Brennan Saeta

commit sha 689791f7903fe530c488ca75682ab4ce5b839002

Add support for booleans in sniffing.

view details

push time in 6 days

push eventsaeta/penguin

Brennan Saeta

commit sha 0dd5fa4634054fb66f6d9fd43c5c7f3a5b8e8955

Update regex comment.

view details

Brennan Saeta

commit sha 459f76de95c6bd6d03360b0c005ab25636ebc3be

Improve accuracy of heuristics in a couple edge cases.

view details

push time in 7 days

push eventsaeta/penguin

Brennan Saeta

commit sha 8182506a4536e8a50baba2c1d50af4c529589a17

To infinity and beyond!

view details

push time in 7 days

push eventsaeta/penguin

Brennan Saeta

commit sha 58f1fceafcbb2a3210223c4564a3a6705dc4a727

Handle non-UTF8 more gracefully.

view details

push time in 7 days

push eventsaeta/penguin

Brennan Saeta

commit sha 99be3c4fa5bd07c89fdbf99c1a4120552620b92f

Add logic to sniff the structure of a csv file.

view details

push time in 7 days

push eventtensorflow/swift-models

Saleem Abdulrasool

commit sha 5ac57baedbdbb948534f143e8cac5ac07192ec3a

Models: use explictly typed math operations (NFC) (#322) Use the typed math operations `powf`, `sqrtf`. This is required to ensure that we can build on Windows.

view details

push time in 8 days

PR merged tensorflow/swift-models

Reviewers
Models: use explictly typed math operations (NFC)

Use the typed math operations powf, sqrtf. This is required to ensure that we can build on Windows.

+2 -2

0 comment

2 changed files

compnerd

pr closed time in 8 days

push eventtensorflow/swift-models

Saleem Abdulrasool

commit sha 8cf419c422fcaf61d9874522207efc13b971560a

BERT: remove hardcoded path assumption (#325) `/tmp` is not the universal temporary directory. Construct the path in a more portable manner by using Foundation. This also allows the user to override the temporary path.

view details

push time in 8 days

PR merged tensorflow/swift-models

Reviewers
BERT: remove hardcoded path assumption

/tmp is not the universal temporary directory. Construct the path in a more portable manner by using Foundation. This also allows the user to override the temporary path.

+3 -1

0 comment

1 changed file

compnerd

pr closed time in 8 days

Pull request review commenttensorflow/swift-models

BERT: remove hardcoded path assumption

 import TensorFlow import TextModels  let bertPretrained = BERT.PreTrainedModel.bertBase(cased: false, multilingual: false)-let workspaceURL = URL(fileURLWithPath: "/tmp/bert_models", isDirectory: true)+let workspaceURL = URL(fileURLWithPath: "bert_models", isDirectory: true,+                       relativeTo: URL(fileURLWithPath: NSTemporaryDirectory(),+                                       isDirectory: true))

At some point, I'd like to have some syntactic sugar for the filesystem. :-( I find this quite cumbersome.

compnerd

comment created time in 8 days

push eventtensorflow/swift-models

Saleem Abdulrasool

commit sha 133b4a979afe0ab978ede947268c481ded32cb9b

Datasets: import FoundationNetworking if possible (#323) This ensures that we import FoundationNetworking whenever available. This is needed when using SwiftFoundation on Darwin or building on Windows.

view details

push time in 8 days

PR merged tensorflow/swift-models

Reviewers
Datasets: import FoundationNetworking if possible

This ensures that we import FoundationNetworking whenever available. This is needed when using SwiftFoundation on Darwin or building on Windows.

+1 -1

0 comment

1 changed file

compnerd

pr closed time in 8 days

Pull request review commenttensorflow/swift-models

move bert demo to examples namespace

 let package = Package(         .target(name: "ModelSupport", dependencies: ["SwiftProtobuf"], path: "Support"),         .target(name: "ImageClassificationModels", path: "Models/ImageClassification"),         .target(name: "TextModels", dependencies: ["Datasets"], path: "Models/Text"),+        .target(name: "Bert-CoLA", dependencies: ["TextModels"], path: "Examples/Bert-CoLA"),

I think you also might need to add the target above in the products section.

Also, nit: Could we call this: BERT-CoLA?

brettkoonce

comment created time in 9 days

Pull request review commenttensorflow/swift-models

Update README.md

 To get a toolchain, you can:  It also currently requires Python 3.x. +The `regex` and `numpy` module are also required for Python.  If this is not available, you can install it via:

Nit: line wrapping.

compnerd

comment created time in 10 days

Pull request review commenttensorflow/swift-apis

Update the AdaGrad optimizer class API docs 📒

 public class RMSProp<Model: Differentiable>: Optimizer  /// AdaGrad optimizer. ///-/// Individually adapts the learning rates of all model parameters by scaling them inversely +/// Implements the AdaGrad optimization algorithm. AdaGrad (adaptive gradient +/// algorithm) has parameter-specific learning rates, which are adapted relative +/// to how frequently a parameter gets updated during training. The more +/// updates a parameter receives, the smaller the learning rate.+///+/// AdaGrad individually adapts the learning rates of all model parameters by scaling them inversely  /// proportional to the square root of the sum of all the historical squared values of the gradient. ///-/// Reference: ["Adaptive Subgradient Methods for Online Learning and Stochastic Optimization"](-/// http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf)+/// Reference: ["Adaptive Subgradient Methods for Online Learning and Stochastic +/// Optimization"](http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf) +/// (Duchi et al, 2011).+///+/// - Parameters:+///     - learningRate: A Float. The learning rate (default value: 1e-3).+///     - rho: A Float. The smoothing factor. Typical values are `0.5`, `0.9`, +///     and `0.99`, for smoothing over 2, 10, and 100 examples, respectively +///     (default value: 0.9).+///     - epsilon: A Float. A small scalar added to the denominator to improve +///     numerical stability (default value: 1e-8).+///+/// ### Examples: ###+/// +/// - Train an image classificaion network:+/// +/// ````+/// ...+/// // Define a sequential model such as LeNet-5.+/// var leNetClassifier = Sequential {+///     Conv2D<Float>(filterShape: (5, 5, 1, 6), padding: .same, activation: relu)+///     AvgPool2D<Float>(poolSize: (2, 2), strides: (2, 2))+///     ...+///     Dense<Float>(inputSize: 84), outputSize: 10)+/// }+/// // Define the AdaGrad optimizer with a learning rate set to 1e-3, rho - to 0.99,+/// // and epsilon: 1e-8+/// let adaGradOptimizer = AdaGrad(for: leNetClassifier, learningRate: 1e-3, +///                                rho: 0.99, epsilon: 1e-8)

Tiny style nit: we might want to have each argument (and label) be on their own line:

e.g.:

/// let adaGradOptimizer = AdaGrad(
///     for: leNetClassifier,
///     learningRate: 1e-3, 
///     rho: 0.99,
///     epsilon: 1e-8)

(For context, the "rule" is spelled out in https://google.github.io/swift/#line-wrapping rule no. 2 that starts "Comma-delimited lists are only laid out in one direction: horizontally or vertically.")

8bitmp3

comment created time in 10 days

push eventtensorflow/swift-models

Saleem Abdulrasool

commit sha 4bae685aa92c02083fb26df4ce3af430f5f4fa7a

Transformer: explicitly indicate encoding (#313) When using this on Windows, Python would complain about the UTF-8 encoding in the file: ``` Non-ASCII character '\\xc2' in file ./Transformer\\encoder.py on line 23, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details ``` Explicitly indicate the encoding in the file.

view details

push time in 10 days

PR merged tensorflow/swift-models

Transformer: explicitly indicate encoding

When using this on Windows, Python would complain about the UTF-8 encoding in the file:

Non-ASCII character '\\xc2' in file ./Transformer\\encoder.py on line 23, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

Explicitly indicate the encoding in the file.

+1 -0

0 comment

1 changed file

compnerd

pr closed time in 10 days

pull request commenttensorflow/swift-apis

Improve precondition error messages.

@t-ae do you think this will be a good fix for #642 ? Thanks!

saeta

comment created time in 10 days

PR opened tensorflow/swift-apis

Improve precondition error messages.

This change makes 2 changes:

  1. It improves the error messages for mis-specified axes.
  2. It ensures the same axis isn't specified multiple times. (Fixes #642.)
+92 -26

0 comment

2 changed files

pr created time in 10 days

create barnchtensorflow/swift-apis

branch : add-preconditions

created branch time in 10 days

create barnchsaeta/penguin

branch : atomics

created branch time in 10 days

issue commenttensorflow/swift-models

UNet Example

We'd love for someone to open a PR with a working UNet example! :-)

tanmayb123

comment created time in 11 days

issue closedtensorflow/swift-models

Add citation requirement to datasets

We should consider adding a soft requirement for citations. I would suggest adding an optional value var citation: String? { get } to ImageClassificationDataset. This should be optional because not every dataset has a citation available.

I'm willing to create a PR for this if the team agrees.

closed time in 11 days

rickwierenga

issue commenttensorflow/swift-models

Add citation requirement to datasets

Hi @rickwierenga ! I think for now, we'll default to have citations in the doc comments for the datasets, but to keep things simple for now, we'll not yet add a citation field in the protocol requirements. Thanks @rickwierenga !

rickwierenga

comment created time in 11 days

push eventtensorflow/swift-apis

8bitmp3

commit sha a8280b455744eedf61a5aaa3939ca6c79f4eda51

Update the Adam optimizer class API docs (#664)

view details

push time in 11 days

PR merged tensorflow/swift-apis

Update the Adam optimizer class API docs

This PR updates the Adam optimizer class API docs, as discussed with @saeta. API docs are important and a few members of the OS community have already looked at things like Sequential and losses.

Introducing a few things to assist the s4tf users:

  • Expanded the description of Adam to be closer to the one in the TF 2.x docs (thanks @lamberta and the docs team).
  • Defined parameters.
  • Added two examples: RL and GAN code snippets adapted from /swift-models/ (Gym and GAN examples).

I'm going through other commonly used classes, funcs, structs and protocols (thought Adam would be a good start), so will be sending more PRs.

Your comments, suggestions, bug fixes are most welcome.

Thanks for all the hard work ☕️

+88 -4

1 comment

1 changed file

8bitmp3

pr closed time in 11 days

delete branch tensorflow/swift-apis

delete branch : ematejska-TF-1150

delete time in 11 days

push eventtensorflow/swift-apis

ematejska

commit sha 479f7422c30dcd1eafef34beeceef08907f21f9e

Revert precondition changes to reshaped (#667) Revert precondition changes to reshaped because it exposed a bug where the shape of the tensor is not available. See: https://bugs.swift.org/browse/TF-1150.

view details

push time in 11 days

PR merged tensorflow/swift-apis

Revert precondition changes to reshaped

Revert precondition changes to reshaped because it exposed a bug where the shape of the tensor is not available. See: https://bugs.swift.org/browse/TF-1150.

+0 -16

3 comments

1 changed file

ematejska

pr closed time in 11 days

pull request commenttensorflow/swift-apis

Update the Adam optimizer class API docs

Thank you @8bitmp3 !!

8bitmp3

comment created time in 13 days

Pull request review commenttensorflow/swift-models

docs: document CMake support

 swift build swift test ``` +### *Experimental* CMake Support++There is experimental support for building with CMake.  This is required to build the models on Windows, and can also be used to cross-compile the models and the demo programs.++**NOTE**: tests are currently not supported with the CMake based build.

Note: this is definitely not ideal. :-(

compnerd

comment created time in 13 days

Pull request review commenttensorflow/swift-models

docs: document CMake support

 swift build swift test ``` +### *Experimental* CMake Support++There is experimental support for building with CMake.  This is required to build the models on Windows, and can also be used to cross-compile the models and the demo programs.++**NOTE**: tests are currently not supported with the CMake based build.++It is highly recommended that you use CMake 3.16 or newer to ensure that `-B` and parallel builds function properly in the example commands below.++macOS:

Does this also work on Linux?

compnerd

comment created time in 13 days

Pull request review commenttensorflow/swift-models

docs: document CMake support

 swift build swift test ``` +### *Experimental* CMake Support++There is experimental support for building with CMake.  This is required to build the models on Windows, and can also be used to cross-compile the models and the demo programs.++**NOTE**: tests are currently not supported with the CMake based build.++It is highly recommended that you use CMake 3.16 or newer to ensure that `-B` and parallel builds function properly in the example commands below.++macOS:++```+cmake                                                              \+  -B /BinaryCache/tensorflow-swift-models                          \

Building into a folder on the root partition is probably a bad idea?

compnerd

comment created time in 13 days

push eventsaeta/penguin

Brennan Saeta

commit sha bb5a289d218fc259b01d37baa835b150ddbea1a5

Limit depth of recursion. (Improves performance by 2x on some benchmarks.)

view details

push time in 14 days

push eventsaeta/penguin

Brennan Saeta

commit sha f98bd6698f6529fb9581767cb119823db0998f4f

Rejigger the pointers to simplify the contexts

view details

push time in 14 days

startedmigueldeicaza/TermKit

started time in 15 days

pull request commentapple/swift

Remove quasiquotes from the tensorflow branch

Thanks @burmako !

burmako

comment created time in 23 days

delete branch tensorflow/swift

delete branch : macosx-toolchain

delete time in 23 days

push eventtensorflow/swift-apis

Rick Wierenga

commit sha 8e325a5f90f61af12c101eee565a314fde035e82

Add documentation to Sequential (#653)

view details

push time in 23 days

PR merged tensorflow/swift-apis

Add documentation to Sequential

See https://github.com/tensorflow/swift-apis/pull/648. The autoencoder example is copied from here. I trust that is OK with you since the source is also a TensorFlow library.

+62 -2

0 comment

2 changed files

rickwierenga

pr closed time in 23 days

PR closed tensorflow/swift-apis

Reviewers
Update README with Sequential API

This also shows commits of an earlier PR for some reason.

+7 -23

3 comments

2 changed files

rickwierenga

pr closed time in 25 days

pull request commenttensorflow/swift-apis

Update README with Sequential API

Hi @rickwierenga ! First of all, thank you so much for this PR. Documentation is super important and I really appreciate you putting in this effort.

I would like our "front-door" README.md to document the "canonical" way to define layers, as this is where people start in their introduction. Because Sequential is fancy / magical syntactic sugar, it might give people the wrong starting point. (Sequential cannot be the canonical way to write layers, because it does not support skip connections, and other sorts of advanced structure.) That said, it'd be great to document Sequential better, and it looks like there's room for improvement. Do you think you could instead turn this change into an example added onto the Sequential type? (Note: you'll want to add it to the .gyb file so your improvements won't be lost.)

To simplify history, I'm going to close this PR, and look forward to your new PR! (Feel free to reference this PR in your new PR.)

All the best, -Brennan

rickwierenga

comment created time in 25 days

push eventsaeta/s4tf-dev-summit-19

Brennan Saeta

commit sha 5437a52df7b5ce3d3e7545ef7e768aac2fbf7966

Fix typo. :-(

view details

push time in a month

push eventsaeta/s4tf-dev-summit-19

Brennan Saeta

commit sha 44355f74180579e7510aa0995244fe2143123c2f

Fix JSON... probably shouldn't hand-edit. Oh well.

view details

push time in a month

push eventsaeta/s4tf-dev-summit-19

Brennan Saeta

commit sha db331a1ab5614d9712daca759439cf8abed203e7

Disable completions. :-(

view details

push time in a month

push eventsaeta/s4tf-dev-summit-19

Brennan Saeta

commit sha 6a8ae8fe261b7f92621c819fd611aee3b341cd04

Update for latest version of S4TF.

view details

push time in a month

push eventsaeta/s4tf-dev-summit-19

Brennan Saeta

commit sha a563e823dff78e0830238e47a23233f16c980167

Update notebook demo for latest version of S4TF in colab.

view details

push time in a month

push eventsaeta/penguin

Brennan Saeta

commit sha f906ab9a4332faa55a30e050f252024d71c38a64

Update documentation for PenguinParallel.

view details

push time in a month

push eventsaeta/penguin

Brennan Saeta

commit sha a8b5db06c80691096b84c34d6abc52f528417643

Rejigger pSum & pMap

view details

push time in a month

push eventsaeta/penguin

Brennan Saeta

commit sha 93013f5b856a974ed4042776b0742842233da825

Refactor to introduce the `PipelineSequence` protocol.

view details

push time in a month

push eventsaeta/penguin

Brennan Saeta

commit sha 56b9be8849041096d146502ca40210c4627a8ebd

Rename `prun` to `pRun` and `pjoin` to `pJoin`.

view details

push time in a month

push eventsaeta/penguin

Brennan Saeta

commit sha 3ef12dc7a58447ebad4f75b79349f6e5d37bb15b

Update documentation for PenguinParallel and clean up a few abstractions.

view details

push time in a month

push eventsaeta/penguin

Brennan Saeta

commit sha 7a78ecd9ca004118a1cd8c2296f2200c49a94f27

Implement pmap on Array. In the process, I debugged actual parallel execution. Note that for short-term correctness, I've added a bunch of extra locks. This is because the focus is not on performance in the short term, but rather to ensure correctness. Note: I'm also checking in the testing executable that I used to test with lldb. This will be deleted in a subsequent commit.

view details

Brennan Saeta

commit sha 56f859c1713d908887aaf4c06ea6fae60e252500

Slight performance tweaks.

view details

Brennan Saeta

commit sha 58b834bae9c8db63081509f34ac35e92edff8c5e

Define the `ThreadPool` protocol and reimplement pjoin in terms of that. This is a much better structure for better composition / etc.

view details

push time in a month

issue commenttensorflow/swift-models

Why is ResNet fixed to only two datasets?

@rickwierenga I entirely agree with @BradLarson and think that this repository should have multiple libraries within it, some focused on data loading, some pre-built models, and some that depend on both to make higher-level abstractions and tools (and of course to do integration testing too). I really like https://github.com/tensorflow/datasets and think a Swift-flavored variation of that would fit in very well in this (swift-models) repository. (Concretely, having fewer repositories means it's easier to see how things fit together because there can be a sample program that combines a pre-built model architecture with a pre-built dataset together into an example program.)

In terms of the software stack & architecture, I see a few major layers:

  1. At the "bottom" is extremely broadly applicable concepts, such as automatic differentiation. This is done more at the "language"-level than library level, but it has a number of important library components. Other things that belong here include our metaprogramming explorations, as well as things like KeyPathIterable, etc.
  2. The next more specific abstraction is a high performance tensor library. This was the motivating work behind Layering Swift APIs. Also at this level is generic parallelism abstractions, as well as some other goodies, such as XLA.
  3. The next level above that are "small" deep-learning-specific building blocks, such as layers and optimizers.
  4. Building on all of the previous levels include models and pre-built input pipelines.
  5. I think there's space for even higher-levels of abstraction here, such as a pre-built image processing toolkit.
  6. Finally at the top are the end-to-end applications (that build on either level 4 or level 5).

In general, the way that development has worked well so far is for new code / features to generally start right next to the application. This allows everyone to iterate on the design and see it in context. Then over time it "bubbles up" to the right level in the stack as we determine how broadly applicable it is. This repository is level's 4, 5, and 6, swift-apis is levels 2 and 3, and finally the core swift repository is level 1.

Hope that helps!

rickwierenga

comment created time in 2 months

more