profile
viewpoint

Ask questionsPubSub: grpc Uncaught exception in the SynchronizationContext. Panic! - Could not find LoadBalancer pick_first

  1. Specify the API: PubSub
  2. OS type and version: Docker image gradle:5.2-jdk11-slim (also happens with jdk8 and other images)
  3. Java version: jdk11 and jdk8
  4. google-cloud-java version(s): 1.62.0 and above (tested with 1.65.0, too)

Steps to reproduce

  1. Use google-cloud-pubsub library with version 1.62.0 or above as dependency
  2. Create a subscription in own code (see code example below)

Code example

ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(projectId, subscriptionId);
try (SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create()) {
    subscriptionAdminClient.getSubscription(subscriptionName); // Exception is thrown in this line
}
...

Stack trace

Mar 19, 2019 6:20:52 PM io.grpc.internal.ManagedChannelImpl$1 uncaughtException
SEVERE: [io.grpc.internal.ManagedChannelImpl-1] Uncaught exception in the SynchronizationContext. Panic!
java.lang.IllegalStateException: Could not find LoadBalancer pick_first. The build probably threw away META-INF/services/io.grpc.LoadBalancerProvider
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory$AutoConfiguredLoadBalancer.<init>(AutoConfiguredLoadBalancerFactory.java:74)
        at io.grpc.internal.AutoConfiguredLoadBalancerFactory.newLoadBalancer(AutoConfiguredLoadBalancerFactory.java:45)
        at io.grpc.internal.ManagedChannelImpl.exitIdleMode(ManagedChannelImpl.java:351)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider$1ExitIdleModeForTransport.run(ManagedChannelImpl.java:447)
        at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:101)
        at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:130)
        at io.grpc.internal.ManagedChannelImpl$ChannelTransportProvider.get(ManagedChannelImpl.java:451)
        at io.grpc.internal.ClientCallImpl.start(ClientCallImpl.java:241)
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1.start(CensusTracingModule.java:392)
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1.start(CensusStatsModule.java:689)
        at io.grpc.ForwardingClientCall.start(ForwardingClientCall.java:32)
        at com.google.api.gax.grpc.GrpcHeaderInterceptor$1.start(GrpcHeaderInterceptor.java:95)
        at io.grpc.stub.ClientCalls.startCall(ClientCalls.java:308)
        at io.grpc.stub.ClientCalls.asyncUnaryRequestCall(ClientCalls.java:280)
        at io.grpc.stub.ClientCalls.futureUnaryCall(ClientCalls.java:189)
        at com.google.api.gax.grpc.GrpcDirectCallable.futureCall(GrpcDirectCallable.java:58)
        at com.google.api.gax.grpc.GrpcExceptionCallable.futureCall(GrpcExceptionCallable.java:64)
        at com.google.api.gax.rpc.AttemptCallable.call(AttemptCallable.java:86)
        at com.google.api.gax.rpc.RetryingCallable.futureCall(RetryingCallable.java:63)
        at com.google.api.gax.rpc.RetryingCallable.futureCall(RetryingCallable.java:41)
        at com.google.api.gax.tracing.TracedUnaryCallable.futureCall(TracedUnaryCallable.java:75)
        at com.google.api.gax.rpc.UnaryCallable$1.futureCall(UnaryCallable.java:126)
        at com.google.api.gax.rpc.UnaryCallable.futureCall(UnaryCallable.java:87)
        at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
        at com.google.cloud.pubsub.v1.SubscriptionAdminClient.getSubscription(SubscriptionAdminClient.java:467)
        at com.google.cloud.pubsub.v1.SubscriptionAdminClient.getSubscription(SubscriptionAdminClient.java:420)
        at my.package.MyClass.start(MyClass.java:37)
        at my.package.App$1.run(App.java:31)
        at java.base/java.lang.Thread.run(Thread.java:834)

External references such as API reference guides used

  • None

Any additional information below

The error only occurs since google-cloud-pubsub 1.62.0 . When using google-cloud-pubsub 1.61.0 or below, the error does not occur with the same user code.

googleapis/google-cloud-java

Answer questions wyaeld

@olavloite

When using kotlin gradle build, I've found that a little more tuning is desirable. The normal workaround of turning on the com.github.johnrengelman.shadow plugin, which has similar function to the maven shade plugin has an issue.

If you do the default config, below:

tasks.withType<ShadowJar> {
    mergeServiceFiles ()
}

it will work, but can result in a lot of undesirable files also ending up in the output jar, I'm unsure why. I ended up with 70mb of gradle internals.

However if you do this

tasks.withType<ShadowJar> {
    mergeServiceFiles {
        setPath("META-INF/services")
        include("io.grpc.*")
    }
}

You get the solution without extra pollution.

Would it be possible to get docs updated to make it easier for people to know this? Its an awkward problem to track down the first time it happens.

useful!

Related questions

Dialogflow V2, Springboot Dialogflow Library calls are failing ::: java.lang.IllegalArgumentException: Failed to load any of the given libraries: [netty_tcnative_linux_x86_64, netty_tcnative_linux_x86_64_fedora, netty_tcnative_x86_64, netty_tcnative] hot 2
Pubsub Subscriber initialization crashes JVM on Alpine Linux with libc6-compat hot 1
Google-cloud-language: No such Method Error on gax grpc hot 1
Duplicate classes error when adding dialogflow dependency 0.98.0-alpha along with firebase hot 1
The newest version of google-cloud-speech jar bug report hot 1
Firestore: Quickstart throws IllegalStateException: Could not find policy 'pick_first' hot 1
Firestore: should library handle/retry UNAVAILABLE errors? hot 1
java.lang.NoSuchMethodError on Google PubSub Publisher.newBuilder().build() hot 1
Error reporting from GCE fails due to auth issues hot 1
Dependency conflict with KMS when running Spark code in Dataproc hot 1
Memory leak from netty/Deadline exceeded after 9999961477ns hot 1
BigQuery: Create new Table using schema from json file hot 1
Unable to make GRPC call in 32bit Windows Machine hot 1
Duplicate classes error when adding dialogflow dependency 0.98.0-alpha along with firebase hot 1
Github User Rank List