Also filed as https://youtrack.jetbrains.com/issue/KT-42822 for the K/N runtime.

K/N Workers does not wrap its work items with an autorelease pool. So when coroutine code calls into ObjC/Swift on these worker threads, any autorelease reference leaks unconditionally, e.g., any id or NSObject* return values.

Not sure if this should be sorted at K/N runtime level. But I did notice that in Kotlin Coroutines there are attempts like:

https://github.com/Kotlin/kotlinx.coroutines/blob/1b0e196bc557f03fd9e13b81edb6997bddeb38e7/kotlinx-coroutines-core/native/src/Workers.kt#L48-L50

where workerMain wraps the block in an autoreleasepool.

This isn't enough however, since the pool is drained only after the EventLoop is closed. So seemingly if my thread is long lived, all the autorelease references are effectively leaked for good.

I want to add a potential use case: having an IndexedDB driver for Web (bridged via packages like SQLWeb). IndexedDB unfortunately doesn't have synchronous query API in the standard, and unlike Java, there is no way to do a blocking wait on Promises AFAIK.

Not sure if this is intentional for Dispatchers.Main.immediate — isDispatchNeeded() returns true when called from main thread.

Seems like it should be !invokedImmediately || !isMainThread()?

https://github.com/Kotlin/kotlinx.coroutines/blob/1b0e196bc557f03fd9e13b81edb6997bddeb38e7/kotlinx-coroutines-core/nativeDarwin/src/Dispatchers.kt#L31

The PRAMGA user_version read for determining need of creation & migration is not part of the subsequent write transaction. It seems to be a plausible race condition, which multiple processes might coincidentially make the decision to run the same prime or migration schema simutaneously. Perhaps the write transaction should be applied to the whole migrateIfNeeded() scope?

https://github.com/touchlab/SQLiter/blob/cabde577c417dd5594e08ae9a98fcdc6f374ee0e/SQLiter/src/nativeCommonMain/kotlin/co/touchlab/sqliter/NativeDatabaseConnection.kt#L84-L95

I reckon iOS File Coordination can be used on the database files to provide guarantees that I need, without needing any change in SQLiter. Thanks anyway.

When ReactiveCocoa is imported as a swift package using Xcode 12, it generates following warning deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. This PR fixes the warning by bumping the min. deployment target to iOS 9 when importing this dependency as a swift packages

I've also updated Quick and Nimble dependencies to their latest versions. The ReactiveSwift counterpart PR is here https://github.com/ReactiveCocoa/ReactiveSwift/pull/802.

When ReactiveSwift is imported as a swift package using Xcode 12, it generates following warning deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.0.99. This PR fixes the warning by bumping the min. deployment target to iOS 9 when importing this dependency as a swift packages

I've also updated Quick and Nimble dependencies to their latest versions. The ReactiveCocoa counterpart PR is here https://github.com/ReactiveCocoa/ReactiveCocoa/pull/3712.

The PRAMGA user_version read for determining need of creation & migration is not part of the subsequent write transaction. It seems to be a plausible race condition, which multiple processes might coincidentially run the same prime or migration schema simutaneously. Perhaps the write transaction should be applied to the whole migrateIfNeeded() scope?

https://github.com/touchlab/SQLiter/blob/cabde577c417dd5594e08ae9a98fcdc6f374ee0e/SQLiter/src/nativeCommonMain/kotlin/co/touchlab/sqliter/NativeDatabaseConnection.kt#L84-L95

I noticed that the native driver as is supports only one connection, shared across all threads, for any query outside a transaction.

My SQLDelight use case is ported from a native Swift stack, and there are plenty of irregular background processing of pushed payloads from various channels. Many of them do listen/query the database outside transactions. So the current configuration has a high probability causing the read-only queries on the main thread UI to be intermittently blocked by background activities, which isn't quite ideal.

If having a reader connection pool is not on the table (due to complexity to workaround K/N memory model?), I strongly believe that the native driver should at least give the main thread its own connection. This way UI applications can still reap the benefits of SQLite's MVCC WAL mode.

Given this query:

getAllCharactersWithBuff:
SELECT character.*, buff.*
FROM Character character LEFT JOIN Buff buff ON character.id = buff.characterId
WHERE character.levelId = ?

It feels like the generated model can do better than listing columns combined from Character and Buff as a flat structure, with those from the latter being individual optional properties.

For exmaple, it could be instead:

data class GetAllCharactersWithBuff(
    character: Character,
    buff: Buff?
)

So columns from buff.* can be accessed as one object, instead of being a sea of optional fields.

Coroutine version: 1.3.9-native-mt Kotlin version: 1.4.10

My testing code has been flaky on Kotlin/Native running on macOS 10.15, with an exception being thrown when it fails.

Uncaught Kotlin exception: kotlin.IllegalStateException: Cannot execute task because event loop was shut down

This happens quite regularly when:

1. For each @Test case, I launch coroutines via launchIn() on any non-Main dispatcher. Attempts include Unconfined, Default and newSingleThreadContext().

2. I perform a busy-wait assertion that spins NSRunLoop.mainRunLoop. (should switch to runBlocking I persume, but this is orthogonal to the bug report here)

3. I forgot to cancel the coroutines with @AfterTest after the test case is completed.

4. The coroutines keep stacking up, until the exception is thrown eventually.

5. All subsequent test cases that use the same dispatcher fail, even though they all pass when being run in isolation.

These observations seem to hold — I have tried two different mitigations, and both do suppress the issue:

• Use Dispatchers.Main in iOS

• Proactively cancel all coroutines when a test case is done.

Stack trace:

        at 0   test.kexe                           0x0000000100cee35d kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Throwable.kt:23:37)
        at 1   test.kexe                           0x0000000100ce792b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:23:44)
        at 2   test.kexe                           0x0000000100ce7aeb kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:34:44)
        at 3   test.kexe                           0x0000000100ce7fdb kfun:kotlin.IllegalStateException#<init>(kotlin.String?){} + 91 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/Exceptions.kt:70:44)
        at 4   test.kexe                           0x0000000100ef07fe kfun:kotlinx.coroutines#loopWasShutDown(){}kotlin.Nothing + 222 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/CoroutineContext.kt:25:50)
        at 5   test.kexe                           0x0000000100ef2bbd kfun:kotlinx.coroutines.EventLoopImplPlatform#reschedule(kotlin.Long;kotlinx.coroutines.EventLoopImplBase.DelayedTask){} + 61 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/EventLoop.kt:22:9)
        at 6   test.kexe                           0x0000000100e592cb kfun:kotlinx.coroutines.EventLoopImplBase.rescheduleAllDelayed#internal + 587 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:397:13)
        at 7   test.kexe                           0x0000000100e55aee kfun:kotlinx.coroutines.EventLoopImplBase#shutdown(){} + 270 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:226:9)
        at 8   test.kexe                           0x0000000100ef305c kfun:kotlinx.coroutines.EventLoopImpl#shutdown(){} + 140 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/EventLoop.kt:34:15)
        at 9   test.kexe                           0x0000000100e53fa4 kfun:kotlinx.coroutines.EventLoop#decrementUseCount(kotlin.Boolean){} + 324 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:114:13)
        at 10  test.kexe                           0x0000000100e5407c kfun:kotlinx.coroutines.EventLoop#decrementUseCount$default(kotlin.Boolean;kotlin.Int){} + 124 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/common/src/EventLoop.common.kt:108:5)
        at 11  test.kexe                           0x0000000100eee4c3 kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} + 1299 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Builders.kt:85:20)
        at 12  test.kexe                           0x0000000100ef72f7 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 407 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Workers.kt:49:17)
        at 13  test.kexe                           0x0000000100ef74de kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$33.invoke#internal + 62 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Workers.kt:47:24)
        at 14  test.kexe                           0x0000000100ef753e kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$33.$<bridge-UNN>invoke(){}#internal + 62 (/opt/buildAgent/work/44ec6e850d5c63f0/kotlinx-coroutines-core/native/src/Workers.kt:47:24)
        at 15  test.kexe                           0x0000000100d1c817 WorkerLaunchpad + 183 (/Users/teamcity/buildAgent/work/cae0e6559deed4c4/runtime/src/main/kotlin/kotlin/native/concurrent/Internal.kt:69:54)
        at 16  test.kexe                           0x0000000100e261df _ZN6Worker19processQueueElementEb + 3135
        at 17  test.kexe                           0x0000000100e25586 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
        at 18  libsystem_pthread.dylib             0x00007fff5dcdc109 _pthread_start + 148
        at 19  libsystem_pthread.dylib             0x00007fff5dcd7b8b thread_start + 15

The exception is thrown at this location: https://github.com/Kotlin/kotlinx.coroutines/blob/1696809ac66753da6f6af71372a3bc4d634f7032/kotlinx-coroutines-core/native/src/EventLoop.kt#L21-L22

Given this reduced

val BackgroundWorkScope = CoroutineScope(newSingleThreadContext("..."))

fun request(): Flow<String> {
    val left = BroadcastChannel<String>(CONFLATED)
    val right = BroadcastChannel<String>(CONFLATED)

    return left.asFlow()
        .zip(right.asFlow()) { lValue, rValue -> "${lValue};${rValue}" }
        .flatMapOnce { it -> flowOf("once: ${it}") }
}

request().launchIn(BackgroundWorkScope)

When the Flow created by request() is executed on different single-thread dispatcher, e.g., via launchIn, InvalidMutabilityException is thrown by Kotlin Coroutines internal.

[Error] mutation attempt of frozen kotlin.native.internal.Ref@39d0f08
kotlin.native.concurrent.InvalidMutabilityException: mutation attempt of frozen kotlin.native.internal.Ref@39d0f08
    at 0   AppName                              0x00000001103d224d kfun:kotlin.Throwable#<init>(kotlin.String?){} + 93
    at 1   AppName                              0x00000001103cb29b kfun:kotlin.Exception#<init>(kotlin.String?){} + 91
    at 2   AppName                              0x00000001103cb4eb kfun:kotlin.RuntimeException#<init>(kotlin.String?){} + 91
    at 3   AppName                              0x00000001103fff3b kfun:kotlin.native.concurrent.InvalidMutabilityException#<init>(kotlin.String){} + 91
    at 4   AppName                              0x00000001104017f2 ThrowInvalidMutabilityException + 690
    at 5   AppName                              0x00000001104fb16c MutationCheck + 108
    at 6   AppName                              0x000000011041fa66 kfun:kotlin.native.internal.Ref#<set-element>(1:0){} + 102
    at 7   AppName                              0x00000001105bb850 kfun:kotlinx.coroutines.flow.<no name provided>_1.<no name provided>_1_8.$emit_2_9COROUTINE$196.invokeSuspend#internal + 736
    at 8   AppName                              0x00000001105bbcd0 kfun:kotlinx.coroutines.flow.<no name provided>_1.<no name provided>_1_8.emit_2_9#internal.246 + 320
    at 9   AppName                              0x00000001105a92bb kfun:kotlinx.coroutines.flow.internal.<no name provided>_1.$collect_2$<anonymous>_7COROUTINE$145.invokeSuspend#internal + 6059
    at 10  AppName                              0x00000001103f2f46 kfun:kotlin.coroutines.native.internal.BaseContinuationImpl#resumeWith(kotlin.Result<kotlin.Any?>){} + 758
    at 11  AppName                              0x00000001105cf472 kfun:kotlinx.coroutines.DispatchedTask#run(){} + 2802
    at 12  AppName                              0x0000000110538596 kfun:kotlinx.coroutines.EventLoopImplBase#processNextEvent(){}kotlin.Long + 838
    at 13  AppName                              0x00000001105f66df kfun:kotlinx.coroutines#runEventLoop(kotlinx.coroutines.EventLoop?;kotlin.Function0<kotlin.Boolean>){} + 911
    at 14  AppName                              0x00000001105feac7 kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.start$lambda-0#internal + 407
    at 15  AppName                              0x00000001105fecae kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$33.invoke#internal + 62
    at 16  AppName                              0x00000001105fed0e kfun:kotlinx.coroutines.WorkerCoroutineDispatcherImpl.$start$lambda-0$FUNCTION_REFERENCE$33.$<bridge-UNN>invoke(){}#internal + 62
    at 17  AppName                              0x0000000110401397 WorkerLaunchpad + 183
    at 18  AppName                              0x00000001104ffcaf _ZN6Worker19processQueueElementEb + 3135
    at 19  AppName                              0x00000001104ff056 _ZN12_GLOBAL__N_113workerRoutineEPv + 54
    at 20  libsystem_pthread.dylib             0x00007fff5dcdc109 _pthread_start + 148
    at 21  libsystem_pthread.dylib             0x00007fff5dcd7b8b thread_start + 15

However, Flow.combine() as a drop-in replacement of zip works fine in this scenario.

val BackgroundWorkScope = CoroutineScope(newSingleThreadContext("..."))

fun request(): Flow<String> {
    val left = BroadcastChannel<String>(CONFLATED)
    val right = BroadcastChannel<String>(CONFLATED)

    return left.asFlow()
        .combine(right.asFlow()) { lValue, rValue -> "${lValue};${rValue}" }
        .flatMapOnce { it -> flowOf("once: ${it}") }
}

request().launchIn(BackgroundWorkScope)

We could mitigate this by applying @_disfavoredOverload, although I don't know if it would have any undesired side effect.

I filed https://bugs.swift.org/browse/SR-13453, although what I reproduced doesn't seem to manifest the same way as described by OP.

Looks like a dup of https://github.com/cashapp/sqldelight/issues/1894. Should I close this @AlecStrong?

Build Environment SQLDelight version: 1.4.1 IDE version: IntelliJ IDEA 2020.2.1 (Community Edition) Build #IC-202.6948.69, built on August 24, 2020 Runtime version: 11.0.8+10-b944.31 x86_64 VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o. macOS 10.15.6 GC: ParNew, ConcurrentMarkSweep Memory: 1979M Cores: 16 Non-Bundled Plugins: org.jetbrains.kotlin, com.squareup.sqldelight

Dialect: SQLite 3.24

Describe the bug If I upgrade the SQLDelight Gradle Plugin to 1.4.1, IntelliJ stops indexing any SQLDelight generated sources, highlighting all generated data classes and interfaces as unresolved references. Gradle builds the project successfully, however.

Only if I stay with Gradle Plugin 1.4.0, IntelliJ indexer stays happy and provides highlighting & autocompletion for the generated source.

related: https://bugs.swift.org/browse/SR-6989

That overload resolution bug in the compiler had happened before but was resolved. Seems like it is time to file it again.

It is using a non-reentrant RAS Signal to inject the event into the Floodgate.

Enqueue the events directly to the Floodgate, which makes it reentrant safe.

It is using a non-reentrant RAS Signal to inject the event into the loop internal.

Enqueue the events directly to the Floodgate, which makes it reentrant safe.

Fix the effect not being started in either of these two scenarios:

1. Given the firstValueAfterNil transform, the initial state gives a non-nil transformed value.
2. Given the whenBecomesTrue predicate, the initial state satisfies the predicate.

Fix the effect not being started in either of these two scenarios:

1. Given the firstValueAfterNil transform, the initial state gives a non-nil transformed value.
2. Given the whenBecomesTrue predicate, the initial state satisfies the predicate.

Preface

A Observer<Value, Error> abstract base class is introduced. It will replace Signal<Value, Error>.Observer as the "currency type" for side effects to be registered with RAS event stream types.

There are three main motivations behind the introduction:

Laying the ground for backpressure support

Backpressure support requires every operator to have extra state for demand management, and a new "subscribed/started" message. This means at the very least:

• The current Signal.Observer abstraction has to be altered.
• The signatures of Signal.Event monads backing many unary stream operators, e.g. map, have to be changed, since the barebone level of closure-as-event-sink (Signal.Event) -> Void they are operating at cannot be extended futher to support the new requirements.

Decluttered stack traces

Nominal operator types incidentially improves also the RAS developer experience — symbols in stack traces now clearly attaches to a nominal Observer type, e.g. Operators.Map, while many compiler inserted reabstraction thunks are now gone with the design change.

Before	After

Improve performance and reduce ARC traffic

Some operators like throttle and debounce require states that persist across events. The current design forces them to be declared externally to the (Signal.Event) -> Void event sink, and then be captured by the closure context of the event sink. So ARC traffic has to be incurred on both the closure context and each captured references.

Changes

• The new Observer<Value, Error> base class.
• Signal.Event.Transformation has a altered signature that is now based on Observer<Value, Error>, instead of (Signal.Event) -> Void.
• A set of operator implementation has been converted to nonimal types under the Operators namespace. This includes Operators.Map, Operator.Filter and Operator.CompactMap.

Checklist

• Updated CHANGELOG.md.

Replaced manually enumerated allEvents to allCases from CaseIterable.

Checklist

• [ ] Updated CHANGELOG.md.

Checklist

• [ ] Updated CHANGELOG.md.

We have adopted @propertyWrapper which requires Swift 5.1+.

Checklist

• Updated CHANGELOG.md.

Checklist

• [ ] Updated CHANGELOG.md.

When Floodgate.withValue owns the recursive lock, events enqueued in parallel might be left unprocessed until another enqueue attempt.

This is because Floodgate.process(_:for:) assumes that, when the call is reentrant, the lock owner is already running an event drain loop. This holds true when Floodgate.process(_:for:) recursively calls itself. But this doesn't hold true when Floodgate.process(_:for:) is the indirect caller of Floodgate.process(_:for:).

This PR refactored the code path, and makes sure that all reducerLock holder fulfills their obligation to drain the event queue after having held the lock.

For the same reason as this was done in ReactiveSwift, which is so that test dependencies aren't exposed to clients of this package.

You need to drop sqldelight:sqlite-driver (which is for JVM) as a common sourceset dependency. Perhaps you meant to include sqldelight:runtime?

@martinbonnin

I have reverted the removals of deprecated methods needing @JsName. It has a knock-on effect in apollo-integration Java sources. I think it is better to be dealt with separately.

For example, removing fromCache() means Java users need to switch to Kotlin synthesized getFromCache(), but the Java convention appears to suggest naming it isFromCache().

Perhaps with all the unnecessary bits reverted, now it makes sense to target 2.x/master instead?