Automated Management of Swift Projects
The Swift Compiler As a Package
SDGGiesbrecht/SDGCornerstone 18
The Foundation of the SDG Module Family
SDGGiesbrecht/SDGCommandLine 3
Tools for Implementing a Command Line Interface
Tools for Implementing a Graphical User Interface
An audio(book)-centric Swift wrapper combining MP4Foundation with ID3TagEditor for tagging .mp3, .m4b, and .m4a files
SDGGiesbrecht/SDGMathematics 2
Extended Support for Mathematics
A simple library for reading and editing metadata and chapters of Mp4 and Mp3 audio files
A simple library for tagging and chaptering Mp4 audio files
Automated Caching of Function Results
Pull request review commentapple/swift-package-manager
public final class ManifestLoader: ManifestLoaderProtocol { swiftpmVersion: Versioning.currentVersion.displayString, fileSystem: fileSystem) - if self.useInMemoryCache, let manifest = self.memoryCache[cacheKey] {+ if self.useInMemoryCache, let manifest = self.memoryCache[cacheKey], manifest.url == baseURL {
I think this is fine, we can also just remove the in-memory cache if we are worried that URL is not the only attribute that could potentially be changes (tho right now its the only issue afaict)
comment created time in an hour
PR opened apple/swift-package-manager
Fix an internal that would occur when switching a dependency between two repositories that had the same last path component and identical package manifest contents (happens frequently with forks). This was due to the in-memory manifest cache, and thus only seen in libSwiftPM clients and not in the CLI.
rdar://73462555
Need to figure out a good way to unit test this without actually accessing network.
pr created time in 2 hours
pull request commentapple/swift-package-manager
SR-12050: lldb error: could not build C module 'CNIOBoringSSLShims' (Linux 5.1.3) when debugging
Once the tests pass, this can be merged to main
. I have already opened a 5.4 PR for SourceKit-LSP, which a 5.4 PR of this fix will then require.
comment created time in 2 hours
pull request commentapple/swift-package-manager
SR-12050: lldb error: could not build C module 'CNIOBoringSSLShims' (Linux 5.1.3) when debugging
@swift-ci please smoke test
comment created time in 2 hours
pull request commentapple/swift-package-manager
SR-12050: lldb error: could not build C module 'CNIOBoringSSLShims' (Linux 5.1.3) when debugging
@swift please smoke test
comment created time in 2 hours
pull request commentapple/swift-package-manager
SR-12050: lldb error: could not build C module 'CNIOBoringSSLShims' (Linux 5.1.3) when debugging
SourceKit-LSP fix has now landed. Restarting tests.
comment created time in 2 hours
pull request commentapple/swift-package-manager
Refactor global verbosity value to parameters
@tomerd back to you. After I removed the default value for verbosity, it surfaced even more places where verbosity
parameter is added.
comment created time in 3 hours
Pull request review commentapple/swift-package-manager
Refactor global verbosity value to parameters
public final class BuildOperation: PackageStructureDelegate, SPMBuildCore.BuildS /// The loaded package graph. private var packageGraph: PackageGraph?- /// The stdout stream for the build delegate. let stdoutStream: OutputByteStream + /// Wether the build is in a verbose mode.+ private let isVerbose: Bool
it's akin to what's in ToolWorkspaceDelegate
, would you prefer to change there as well?
comment created time in 4 hours
push eventapple/swift-package-manager
commit sha 95e5c3f6ec31fc3612ba7cc5df495726a740216a
Package collections: improve search performance (#3213) * Package collections: improve search performance This is an alternate to https://github.com/apple/swift-package-manager/pull/3090 but is a complete solution. Motivation: Currently to support search for package collections API we read and deserialize collection blobs from SQLite then perform string matchings on individual properties in memory (e.g., `package.summary.contains("foobar")`). This can be optimized. Modifications: Use SQLite FTS--define FTS virtual tables for packages and targets, and update implementation for `findPackage`, `searchPackages`, and `searchTargets` methods of `SQLitePackageCollectionsStorage`. Without optimization, `PackageCollectionsTests.testPackageSearchPerformance` and `testTargetsSearchPerformance` take about ~400ms to run on my local machine. With FTS, `testPackageSearchPerformance` takes ~40ms and `testTargetsSearchPerformance` ~50ms. The `testSearchTargetsPerformance` in `InMemoryPackageCollectionsSearchTests` (https://github.com/apple/swift-package-manager/pull/3090) yields result in ~10ms, though it queries the trie directly without going through the PackageCollections API layer. Since target search is either exact or prefix match and doesn't tokenize the query, and given the good results I saw in `InMemoryPackageCollectionsSearchTests`, this implementation includes a trie on top of SQLite FTS for target search. The trie is in-memory and loads from SQLite FTS during initialization. `put`/`remove` updates both the SQLite FTS and trie. The improvement with using trie varies--`testTargetsSearchPerformance` takes between ~15-50ms to complete. `put` now takes longer to complete because of the search index updates. Result: Better search performance. * Fallback to slow path, which does not use FTS, in case FTS4 is not available
push time in 4 hours
PR merged apple/swift-package-manager
This is an alternative to https://github.com/apple/swift-package-manager/pull/3090 but is a complete solution.
Motivation:
Currently to support search for package collections API we read and deserialize collection blobs from SQLite then perform string matchings on individual properties in memory (e.g., package.summary.contains("foobar")
). This can be optimized.
Modifications:
Use SQLite FTS--define FTS virtual tables for packages and targets, and update implementation for findPackage
, searchPackages
, and searchTargets
methods of SQLitePackageCollectionsStorage
.
Without optimization, PackageCollectionsTests.testPackageSearchPerformance
and testTargetsSearchPerformance
take about ~400ms to run on my local machine.
With FTS, testPackageSearchPerformance
takes ~40ms and testTargetsSearchPerformance
~50ms.
The testSearchTargetsPerformance
in InMemoryPackageCollectionsSearchTests
(https://github.com/apple/swift-package-manager/pull/3090) yields result in ~10ms, though it queries the trie directly without going through the PackageCollections API layer.
Since target search is either exact or prefix match and doesn't tokenize the query, and given the good results I saw in InMemoryPackageCollectionsSearchTests
, this implementation includes a trie on top of SQLite FTS for target search. The trie is in-memory and loads from SQLite FTS during initialization. put
/remove
updates both the SQLite FTS and trie. The improvement with using trie varies--testTargetsSearchPerformance
takes between ~15-50ms to complete.
put
now takes longer to complete because of the search index updates.
Result: Better search performance.
pr closed time in 4 hours
pull request commentapple/swift-package-manager
SR-12050: lldb error: could not build C module 'CNIOBoringSSLShims' (Linux 5.1.3) when debugging
Talked with @benlangmuir and opened https://github.com/apple/sourcekit-lsp/pull/362 to make SourceKit-LSP's unit test accept both forms of command line until this PR has been merged.
comment created time in 4 hours
Pull request review commentapple/swift-package-manager
api cleanup in preperation for identity improvements
public enum DependencyResolutionNode { // Don’t create a version lock for anything but a product. guard specificProduct != nil else { return nil } return PackageContainerConstraint(- container: package,+ package: package,
Can we call this packageReference
?
comment created time in 5 hours
Pull request review commentapple/swift-package-manager
api cleanup in preperation for identity improvements
import TSCUtility /// should be used as-is. Infact, they might not even have a git repository. /// Examples: Root packages, local dependencies, edited packages. public final class LocalPackageContainer: PackageContainer {- public let identifier: PackageReference+ public let package: PackageReference
Can we call this packageReference
?
comment created time in 5 hours
pull request commentapple/swift-package-manager
SR-12050: lldb error: could not build C module 'CNIOBoringSSLShims' (Linux 5.1.3) when debugging
Looks like I missed a Linux-specific unit test. Investigating.
So, 1) it's not Linux-specific, and 2) it's a SourceKit-LSP test (which is why local unit tests all passed and the self-hosted test passed). Looks as if we have a revlock issue here on the SourceKit-LSP tests. @benlangmuir what's your view on how we should address this, since these are in two different repos and neither change will succeed on its own. Could the SourceKit-LSP tests become a bit less specific about whether we're passing -Xcc
?
comment created time in 5 hours
Pull request review commentapple/swift-package-manager
Package collections: improve search performance
final class SQLitePackageCollectionsStorage: PackageCollectionsStorage, Closable """ try db.exec(query: table) - let ftsPackages = """- CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.packagesFTSName) USING fts4(- collection_id_blob_base64, id, version, name, repository_url, summary, keywords, products, targets,- notindexed=collection_id_blob_base64,- tokenize=unicode61- );- """- try db.exec(query: ftsPackages)-- let ftsTargets = """- CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.targetsFTSName) USING fts4(- collection_id_blob_base64, package_repository_url, name,- notindexed=collection_id_blob_base64,- tokenize=unicode61- );- """- try db.exec(query: ftsTargets)+ do {+ let ftsPackages = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.packagesFTSName) USING fts4(+ collection_id_blob_base64, id, version, name, repository_url, summary, keywords, products, targets,+ notindexed=collection_id_blob_base64,+ tokenize=unicode61+ );+ """+ try db.exec(query: ftsPackages)++ let ftsTargets = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.targetsFTSName) USING fts4(+ collection_id_blob_base64, package_repository_url, name,+ notindexed=collection_id_blob_base64,+ tokenize=unicode61+ );+ """+ try db.exec(query: ftsTargets)++ useSearchIndices.put(true)+ } catch {+ // These DDL statements work but queries yield different results when run on different+ // platforms. This could be because of SQLite version perhaps? But since we can't get+ // consistent results we will not fallback to FTS3 and just give up if FTS4 is not available.+ /*+ let ftsPackages = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.packagesFTSName) USING fts3(+ collection_id_blob_base64, id, version, name, repository_url, summary, keywords, products, targets,+ tokenize=simple+ );+ """+ try db.exec(query: ftsPackages)++ let ftsTargets = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.targetsFTSName) USING fts3(+ collection_id_blob_base64, package_repository_url, name,+ tokenize=porter+ );+ """+ try db.exec(query: ftsTargets)
removed https://github.com/apple/swift-package-manager/pull/3213/commits/a84e786e1d2fab0254c3d21327a0e79eb8f4e642
comment created time in 5 hours
pull request commentapple/swift-package-manager
Package collections: improve search performance
@swift-ci please smoke test
comment created time in 5 hours
pull request commentapple/swift-package-manager
SR-12050: lldb error: could not build C module 'CNIOBoringSSLShims' (Linux 5.1.3) when debugging
Looks like I missed a Linux-specific unit test. Investigating.
comment created time in 6 hours
Pull request review commentapple/swift-package-manager
Refactor global verbosity value to parameters
public final class XcodeBuildSystem: BuildSystem { public init( buildParameters: BuildParameters, packageGraphLoader: @escaping () throws -> PackageGraph,- isVerbose: Bool,+ isVerbose: Bool = false,
as in the other spot, I prefer passing the full Verbosity value and not having a default so call-sites make conscious decision
comment created time in 6 hours
Pull request review commentapple/swift-package-manager
Refactor global verbosity value to parameters
public final class ManifestLoader: ManifestLoaderProtocol { packageKind: packageKind, fileSystem: fileSystem, diagnostics: diagnostics,- on: queue) { result in+ on: queue,+ verbosity: verbosity+ ) { result in
not: align this to the back
comment created time in 6 hours
Pull request review commentapple/swift-package-manager
Refactor global verbosity value to parameters
public final class ManifestLoader: ManifestLoaderProtocol { toolsVersion: ToolsVersion, packageKind: PackageReference.Kind, fileSystem: FileSystem? = nil,- diagnostics: DiagnosticsEngine? = nil+ diagnostics: DiagnosticsEngine? = nil,+ verbosity: Verbosity = TSCUtility.verbosity
this API is deprecated so we should not add argument here, just pass in TSCUtility.verbosity as default in L282
comment created time in 6 hours
Pull request review commentapple/swift-package-manager
Refactor global verbosity value to parameters
extension ManifestLoaderProtocol { fileSystem: FileSystem? = nil, diagnostics: DiagnosticsEngine? = nil, on queue: DispatchQueue,+ verbosity: Verbosity = TSCUtility.verbosity,
I'd rather not have a default set here to make sure the call-site make a conscious decision. For the unit test we can have an short-form via extension
comment created time in 6 hours
Pull request review commentapple/swift-package-manager
Refactor global verbosity value to parameters
public final class BuildOperation: PackageStructureDelegate, SPMBuildCore.BuildS /// The loaded package graph. private var packageGraph: PackageGraph?- /// The stdout stream for the build delegate. let stdoutStream: OutputByteStream + /// Wether the build is in a verbose mode.+ private let isVerbose: Bool
wdyt about storing Verbosity
instead of Bool
as the member, and adding a "isVerbose" computed property that is calculated off of that enum value?
comment created time in 6 hours
Pull request review commentapple/swift-package-manager
Move package collection JSON models to a separate module
struct JSONPackageCollectionProvider: PackageCollectionProvider { extension Model.Product { fileprivate init(from: JSONModel.Product, packageTargets: [Model.Target]) { let targets = packageTargets.filter { from.targets.map { $0.lowercased() }.contains($0.name.lowercased()) }- self = .init(name: from.name, type: from.type, targets: targets)+ self = .init(name: from.name, type: .init(from: from.type), targets: targets)+ }+}++extension PackageModel.ProductType {+ fileprivate init(from: JSONModel.ProductType) {+ switch from {+ case .library(let libraryType):+ self = .library(.init(from: libraryType))+ case .executable:+ self = .executable+ case .test:+ self = .test+ }+ }+}++extension PackageModel.ProductType.LibraryType {+ fileprivate init(from: JSONModel.ProductType.LibraryType) {+ switch from {+ case .static:+ self = .static+ case .dynamic:+ self = .dynamic+ case .automatic:+ self = .automatic+ } }
I assume we dont re-user the package model types for separation of concerns?
comment created time in 6 hours
Pull request review commentapple/swift-package-manager
Move package collection JSON models to a separate module
/* This source file is part of the Swift.org open source project - Copyright (c) 2020 Apple Inc. and the Swift project authors+ Copyright (c) 2020-2021 Apple Inc. and the Swift project authors Licensed under Apache License v2.0 with Runtime Library Exception See http://swift.org/LICENSE.txt for license information See http://swift.org/CONTRIBUTORS.txt for Swift project authors */ +import Foundation+ public enum JSONPackageCollectionModel {}
is the JSON
prefix needed, since we are using Codable it could theoretically some different format? not a strong feeling, more of a question really
comment created time in 7 hours
Pull request review commentapple/swift-package-manager
Package collections: improve search performance
final class SQLitePackageCollectionsStorage: PackageCollectionsStorage, Closable """ try db.exec(query: table) - let ftsPackages = """- CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.packagesFTSName) USING fts4(- collection_id_blob_base64, id, version, name, repository_url, summary, keywords, products, targets,- notindexed=collection_id_blob_base64,- tokenize=unicode61- );- """- try db.exec(query: ftsPackages)-- let ftsTargets = """- CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.targetsFTSName) USING fts4(- collection_id_blob_base64, package_repository_url, name,- notindexed=collection_id_blob_base64,- tokenize=unicode61- );- """- try db.exec(query: ftsTargets)+ do {+ let ftsPackages = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.packagesFTSName) USING fts4(+ collection_id_blob_base64, id, version, name, repository_url, summary, keywords, products, targets,+ notindexed=collection_id_blob_base64,+ tokenize=unicode61+ );+ """+ try db.exec(query: ftsPackages)++ let ftsTargets = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.targetsFTSName) USING fts4(+ collection_id_blob_base64, package_repository_url, name,+ notindexed=collection_id_blob_base64,+ tokenize=unicode61+ );+ """+ try db.exec(query: ftsTargets)++ useSearchIndices.put(true)+ } catch {+ // These DDL statements work but queries yield different results when run on different+ // platforms. This could be because of SQLite version perhaps? But since we can't get+ // consistent results we will not fallback to FTS3 and just give up if FTS4 is not available.+ /*+ let ftsPackages = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.packagesFTSName) USING fts3(+ collection_id_blob_base64, id, version, name, repository_url, summary, keywords, products, targets,+ tokenize=simple+ );+ """+ try db.exec(query: ftsPackages)++ let ftsTargets = """+ CREATE VIRTUAL TABLE IF NOT EXISTS \(Self.targetsFTSName) USING fts3(+ collection_id_blob_base64, package_repository_url, name,+ tokenize=porter+ );+ """+ try db.exec(query: ftsTargets)
should we remove this? or keep around so we can fix later?
comment created time in 7 hours
pull request commentapple/swift-package-manager
Unless you plan on using these new crypto features when building SPM with itself, which shouldn't be necessary since build-script
should have already checked out SPM's dependencies, I think it's fine to not add this swift-crypto dependency for the CMake bootstrap and avoid adding a CMake config for swift-crypto.
comment created time in 7 hours
PR opened apple/swift-package-manager
Refactor global TSCUtils.verbosity
value to functions parameter
Motivation:
While it's okayish to maintain global TSCUtils.verbosity
for CLI tool, it's not really for library API. I believe it's better to provide a configurable verbosity parameter for library public API.
Modifications:
replace TSCUtils.verbosity
with function parameter that default to global TSCUtils.verbosity
Result:
API verbosity setup is more granulated.
pr created time in 9 hours
push eventapple/swift-syntax
commit sha b227790692b73cb494d9f47a00fac2f4b824b88a
Remove advancedToEndOfChildren These methods are not used anywhere and are potentially incorrect (indexInParent should probably be raw.numberOfChildren - 1). Remove the until we really need them
commit sha 1e998e808e3ff7c28c6f7d33c4480dbac0a7040a
Rename advancedBy/reversedBySibling to advancedBy/reversedBy We are not actually advancing by a sibling but just by a node, simplify the method name to reflect this.
commit sha d8aa974f0532d7f0ce42b87afe476d5e4e9211ab
Merge pull request #255 from ahoppen/api-improvals Minor internal API improvements
push time in 17 hours
PR merged apple/swift-syntax
Two minor internal API improvements:
- Remove
advancedToEndOfChildren
These methods are not used anywhere and are potentially incorrect (indexInParent
should probably be initialized withraw.numberOfChildren - 1
). Remove the until we really need them - Rename
advancedBySibling
/reversedBySibling
toadvancedBy
/reversedBy
We are not actually advancing by a sibling but just by a node, simplify the method name to reflect this.
pr closed time in 17 hours
pull request commentapple/swift-package-manager
Package collections: improve search performance
@swift-ci please smoke test
comment created time in 17 hours