Currently, callback logic for methods that return a publisher is implemented by wrapping the result publisher with a new Flux using doOnSubscribe, doFinally, etc., and same for query callback. Since this is a bit fragile implementation, instead, create dedicated custom subscribers that performs callback logic, and apply them to the returned publisher.

Currently, callback logic for methods that return a publisher is implemented by wrapping the result publisher with a new Flux using doOnSubscribe, doFinally, etc., and same for query callback. Since this is a bit fragile implementation, instead, create dedicated custom subscribers that performs callback logic, and apply them to the returned publisher.

Yeah, I have a utility class that uses the String key "org.springframework.security.SECURITY_CONTEXT_ATTRIBUTES" to retrieve Authentication from reactor context. This works but since it is bit of internal to the spring security, it would be nice to have public(official) way of getting Authentication(as well as HttpServletRequest and HttpServletResponse) in WebClient filters in servlet environment. I think it is in common needs getting Authentication when writing a custom WebClient filters.

Currently, there is no means to retrieve ProxyConfig from proxied objects. Similar to the ConnectionHolder interface which provides a method to extract Connection from proxied objects, introduce ProxyConfigHolder interface and add handling logic to the proxy handler.

Currently, there is no means to retrieve ProxyConfig from proxied objects. Similar to the ConnectionHolder interface which provides a method to extract Connection from proxied objects, introduce ProxyConfigHolder interface and add handling logic to the proxy handler.

We can aim to make it as easy as possible but we can't make the assumption that it is always needed.

Ok, thanks. At least having mechanism to pass values to nested calls is really helpful. Then, WebClientCustomizer can handle population part in boot application.

@quaff hm, by quick looking the code, what I can think of are:

• Original definition is looked up via bean name dataSource, so it won't work when DataSource is registered under different name.
• Cannot simply retrieve the raw DataSource. It always need to unwrap from TracingDataSource.
• For DataSource, if connection pool is involved, which usually shaped as DataSource and delegate to actual DataSource, usually what you want to instrument is the outer most DataSource because you want to capture the application behavior not the connection pool behavior. (can be but secondary to the application behavior). I think swapping bean definitions would be difficult to handle such case.
• It is using GlobalTracer.get() as static retrieval. but you want to allow configuring own tracer bean to use. So, dependency injection would be hard to work on at this low level.

I think my solution is simpler and elegant

Haha, yeah, probably. It's subjective and also depends on the requirement of your application. For example, sleuth uses more BPP and open-telemetry maybe use byte code manipulation(I didn't check, simply my guess since I remember I saw ByteBuddy dependency there) for tracing instrumentation. I think one drawback with BFPP impl above is it will lose the original bean definition with original name. So, whether it is acceptable compromise or not is upto the requirement for your app.

Sorry, it has derailed the original issue. Just a bit of side chatting.

Hi @quaff

Isn't it more natural to use BeanPostProcessor to decorate those beans?

For example, spring-boot-data-source-decorator wraps DataSource with BPP here.

BTW, if you want to instrument DataSource with tracer, you could use spring-boot-data-source-decorator to wrap DataSource with datasource-proxy or p6spy for Spring Boot application. Or you could use java-jdbc if it is opentracing.

When [Mono|Flux]#usingWhen is used for ConnectionFactory#create(), afterMethod callback is performed AFTER Connection::close(from async-cleanup).

Flux.usingWhen(connectionFactory.create(), connection -> {
    // perform db operation
}, Connection::close, (c, err) -> c.close(), Connection::close);

This is because usingWhen method defers the completion of "resource supplier"(ConnectionFactory#create) until "async-cleanup"(Connection#close) to be finished.

There are two ways in r2dbc-proxy to make this work:

• Wrap the proxied result(Flux) to Mono. When first element is emitted, the Mono makes cancel to the wrapped Flux which triggers doFinally on the Flux before "async-cleanup".

• Use doOnNext to trigger afterMethod callback. Instead of using doFinally, special handle the ConnectionFactory#create to use doOnNext() to trigger the afterMethod callback. As soon as an element is emitted, doOnNext() is called before "async-cleanup".

If client side need to make it work for now, the quick workaround is:

Wrap the ConnectionFactory#create with Mono#from

return Flux.usingWhen(Mono.from(this.connectionFactory.create()), connection -> {
  // logic to access DB
}, Connection::close, (c, err) -> c.close(), Connection::close);

For DatabaseClient in spring, the logic to acquire a connection already wraps the ConnectionFactory#create with Mono; so, DatabaseClient works without any changes.

private static Mono<Connection> fetchConnection(ConnectionFactory connectionFactory) {
  return Mono.from(connectionFactory.create());
}

add an option to have ClientRequest attributes be saved under some well known key in the Reactor context.

This sounds a nice solution.

It may also be good to auto attach servlet request, etc to the reactor context if in servlet environment; So that, ExchangeFilterFunction can have consistent way to retrieve servlet request and values associated to the attributes. This gives benefit to library writers since everything concludes in ExchangeFilterFunction, maybe in combination with servlet Filter to populates values to request attributes, instead of interleaving WebClient.Builder or WebClient#attributes at WebClient build time.

If WebClient.Builder could setup its subscriber context(defaultContext method below), maybe this would work.

FooContext.set("main-thread");  // setting to threadlocal

// a filter uses "foo" from subscriber context
ExchangeFilterFunction filter = (request, next) ->
    Mono.subscriberContext()
        .map(context -> (String) context.get("foo"))
        .map(foo -> ClientRequest.from(request).header("foo", foo).build())
        .flatMap(next::exchange);


WebClient webClient = WebClient.builder()
        .filter(filter)
        // populate subscriber context
        .defaultContext(context -> context.hasKey("foo") ? context : context.put("foo", FooContext.get()))
        .build();

webClient.get()
        .uri("http://foo.com")
        .retrieve()
        .bodyToMono(String.class)
        .flatMap(body ->
                webClient.get()
                        .uri("http://bar.com")
                        .retrieve()
                        .bodyToMono(String.class)
        )
        .subscribeOn(Schedulers.boundedElastic())  // on different thread...
        ....
;

// both calls will have header "foo=main-thread"

When nested WebClient is called, the context key "foo" has already populated by outer WebClient, which used the original caller's thread.

Not sure it is even possible, but an idea.

When [Mono|Flux]#usingWhen is used for ConnectionFactory#create(), afterMethod callback is performed AFTER Connection::close(from async-cleanup).

Flux.usingWhen(connectionFactory.create(), connection -> {
    // perform db operation
}, Connection::close, (c, err) -> c.close(), Connection::close);

This is because usingWhen method defers the completion of "resource supplier"(ConnectionFactory#create) until "async-cleanup"(Connection#close) to be finished.

There are two ways in r2dbc-proxy to make this work:

• Wrap the proxied result(Flux) to Mono. When first element is emitted, the Mono makes cancel to the wrapped Flux which triggers doFinally on the Flux before "async-cleanup".

• Use doOnNext to trigger afterMethod callback. Instead of using doFinally, special handle the ConnectionFactory#create to use doOnNext() to trigger the afterMethod callback. As soon as an element is emitted, doOnNext() is called before "async-cleanup".

If client side need to make it work for now, the quick workaround is:

Wrap the ConnectionFactory#create with Mono#from

return Flux.usingWhen(Mono.from(this.connectionFactory.create()), connection -> {
  // logic to access DB
}, Connection::close, (c, err) -> c.close(), Connection::close);

For DatabaseClient in spring, the logic to acquire a connection already wraps the ConnectionFactory#create with Mono; so, DatabaseClient works without any changes.

private static Mono<Connection> fetchConnection(ConnectionFactory connectionFactory) {
  return Mono.from(connectionFactory.create());
}

@rstoyanchev Oh, I wasn't aware of attributes method. Thanks. Certainly it would be nice to mention ThreadLocal in documentation for attribute method usage.

I have simulated couple of our usecases trying with attributes in servlet environment with WebClient. This serves most of the cases since I can assign values to attributes from Threadlocal at the time when request is made and on the thread that is making a request.

However, I found one case hard to do with attributes, which is nesting remote calls. For example, make a first remote call, then based on the response, create another remote call.

Sample code:

ExchangeFilterFunction filter = (request, next) -> {
  log.info("filter map=" + request.attributes());
  return next.exchange(request);
};

WebClient webClient = WebClient.builder().filter(filter).defaultRequest(spec -> {
  spec.attribute("bar", "BAR");
  spec.attribute("tid", Thread.currentThread().getId());
  spec.attribute("tname", Thread.currentThread().getName());
}).build();

webClient.get().uri("http://service1.com")  // first call
    .attribute("foo", "FOO")
    .retrieve()
    .bodyToMono(String.class)
    .flatMap(str -> {
      return webClient.get().uri("http://service2.com")  // second call
          .retrieve()
          .bodyToMono(String.class);
    })
    .subscribeOn(Schedulers.boundedElastic())  // on different thread...
    .block();

When making the first call, attributes tid and tname are main thread, but for the second call, the thread is on elastic thread; so cannot retrieve thread local values that are bound to the caller.

In the real environment, these two calls are service-to-service calls and when I call another service, I need to propagate some values stored in caller's thread local. (very much similar to ServletBearerExchangeFilterFunction in Spring Security)

Workaround can be, when constructing the second call, explicitly add attribute method and pass a value that the filter is looking for. However, it is invasive to application code and would like to handle it in filter or construction of webclient in library.

Currently, I use subscriber context approach as mentioned in the description. (reusing SecurityReactorContextSubscriberRegistrar from Spring Security)

Also, interesting thing is that the ServletOAuth2AuthorizedClientExchangeFilterFunction (pointed in your comment) uses attributes but also it is getting attribute values(request, response, and auth) from subscriber context which is populated by the lifter hook (SecurityReactorContextSubscriberRegistrar).

So, attributes works for single remote call cases, but when multiple calls on different threads are involved, I think above mentioned Hook approach may be in need.

I believe priority ordered BFPPs need to be applied first, THEN rest of the BFPP can be instantiated.

For those who want to have auto configuration order logged: I wrote this class as a temporal workaround to write out "Auto configuration order report" to the log. This class reflectively accesses auto configuration sorter and write the report upon receiving AutoConfigurationImportEvent.

code:

/**
 * Write the processing order of auto configuration classes to the log.
 * <p>
 * This implementation reflectively invokes {@code AutoConfigurationSorter} to sort auto
 * configuration classes since Spring Boot does not expose the sorting infrastructure for
 * auto configuration classes.
 * <p>
 * Note: This class doesn't work with {@code ApplicationContextRunner}. This is because
 * {@code ApplicationContextRunner} does not trigger
 * {@link AutoConfigurationImportListener} event and relies on the directly
 * populating(recording) {@link ConditionEvaluationReport} via
 * {@link SpringBootCondition}. The configuration order is sorted by
 * {@code AutoConfigurations} when new configurations are added to the runner. One
 * possible solution is to use {@code ApplicationContextRunner#with} method to return a
 * proxy. At the time of running the runner, it can introspect all configurations, which
 * has ordered, and report the order of auto configuration classes.
 * <p>
 * https://github.com/spring-projects/spring-boot/issues/20732
 *
 * @author Tadaya Tsuyukubo
 */
// Inherit the logging category from Spring Boot. For example, with "--debug" option,
// "LoggingApplicationListener" sets the log level to "debug" for Spring Boot categories.
// Then, this logger will automatically inherit the same level.
@Slf4j(topic = "org.springframework.boot.mylib")
public class AutoConfigurationOrderLoggingListener
		implements BeanFactoryAware, ResourceLoaderAware, AutoConfigurationImportListener {

	private static final String SORTER_CLASS = "org.springframework.boot.autoconfigure.AutoConfigurationSorter";

	private ConfigurableListableBeanFactory beanFactory;

	private ResourceLoader resourceLoader;

	@Override
	public void onAutoConfigurationImportEvent(AutoConfigurationImportEvent event) {
//		if (!log.isDebugEnabled()) {
//			return; // only run when debug is on
//		}

		List<String> configs = event.getCandidateConfigurations();
		List<String> sorted;
		try {
			sorted = sort(configs);
		}
		catch (Exception ex) {
			log.error("Failed to sort auto configuration classes", ex);
			return; // do not proceed
		}

		// If we want to follow how "ConditionEvaluationReport" is logged, log them at
		// "ContextRefreshedEvent" and "ApplicationFailedEvent" with implementing
		// "GenericApplicationListener" (see "ConditionEvaluationReportLoggingListener").
		// Then, register to "spring.factories" as "ApplicationListener".
		logAutoConfigurationOrder(sorted);
	}

	private void logAutoConfigurationOrder(List<String> configs) {
		StringBuilder sb = getLogMessage(configs, "AUTO CONFIGURATION ORDER REPORT");
		log.debug(sb.toString());
	}

	private StringBuilder getLogMessage(List<String> configs, String title) {
		StringBuilder message = new StringBuilder();
		message.append(String.format("%n%n%n"));
		StringBuilder separator = new StringBuilder();
		for (int i = 0; i < title.length(); i++) {
			separator.append("=");
		}
		message.append(String.format("%s%n", separator));
		message.append(String.format("%s%n", title));
		message.append(String.format("%s%n%n%n", separator));
		for (int i = 0; i < configs.size(); i++) {
			message.append(String.format("%3d - %s%n", i + 1, configs.get(i)));
		}
		message.append(String.format("%n%n"));
		return message;
	}

	// Equivalent to
	// "org.springframework.boot.autoconfigure.AutoConfigurationImportSelector.AutoConfigurationGroup.sortAutoConfigurations"
	@SuppressWarnings("unchecked")
	private List<String> sort(List<String> configs) throws Exception {

		Class<?> clazz = ClassUtils.resolveClassName(SORTER_CLASS,
				AutoConfigurationOrderLoggingListener.class.getClassLoader());
		Method method = ReflectionUtils.findMethod(clazz, "getInPriorityOrder", Collection.class);
		ReflectionUtils.makeAccessible(method);

		Constructor<?> constructor = ReflectionUtils.accessibleConstructor(clazz, MetadataReaderFactory.class,
				AutoConfigurationMetadata.class);
		Object[] args = new Object[] { getMetadataReaderFactory(), getAutoConfigurationMetadata() };

		Object sorter = BeanUtils.instantiateClass(constructor, args);

		List<String> sorted = (List<String>) ReflectionUtils.invokeMethod(method, sorter, configs);
		return sorted;
	}

	// Equivalent to
	// "org.springframework.boot.autoconfigure.AutoConfigurationImportSelector.AutoConfigurationGroup.getMetadataReaderFactory"
	private MetadataReaderFactory getMetadataReaderFactory() {
		String beanName = "org.springframework.boot.autoconfigure.internalCachingMetadataReaderFactory";
		try {
			return this.beanFactory.getBean(beanName, MetadataReaderFactory.class);
		}
		catch (NoSuchBeanDefinitionException ex) {
			return new CachingMetadataReaderFactory(this.resourceLoader);
		}
	}

	// Equivalent to
	// "org.springframework.boot.autoconfigure.AutoConfigurationImportSelector.AutoConfigurationGroup.getAutoConfigurationMetadata"
	private AutoConfigurationMetadata getAutoConfigurationMetadata() {
		String loaderClass = "org.springframework.boot.autoconfigure.AutoConfigurationMetadataLoader";
		Class<?> clazz = ClassUtils.resolveClassName(loaderClass,
				AutoConfigurationOrderLoggingListener.class.getClassLoader());
		Method method = ReflectionUtils.findMethod(clazz, "loadMetadata", ClassLoader.class);
		ReflectionUtils.makeAccessible(method);
		AutoConfigurationMetadata metadata = (AutoConfigurationMetadata) ReflectionUtils.invokeMethod(method, null,
				AutoConfigurationOrderLoggingListener.class.getClassLoader());
		return metadata;
	}

	@Override
	public void setBeanFactory(BeanFactory beanFactory) throws BeansException {
		Assert.isInstanceOf(ConfigurableListableBeanFactory.class, beanFactory);
		this.beanFactory = (ConfigurableListableBeanFactory) beanFactory;
	}

	@Override
	public void setResourceLoader(ResourceLoader resourceLoader) {
		this.resourceLoader = resourceLoader;
	}

}

spring.factories:

org.springframework.boot.autoconfigure.AutoConfigurationImportListener=\
com.example.AutoConfigurationOrderLoggingListener

We have an operational scenario that directly open a shell on the pod and restart the application. This allows faster restart of the app since it doesn't involve k8s pod lifecycle.

In such case, k8s does not know the application is restarting and it still send traffic to the pod during shutdown. So, in our graceful shutdown logic(in addition to the one from boot 2.3), we put a sleep for the duration of readiness check frequency, to make sure k8s readiness check picks up that the app is about to restarting and stop routing the traffic before actual shutdown happens.

If Spring Boot provides sleep(delay) on its graceful shutdown logic, I can remove our sleep logic from our library, which is great.

We use WebClient in servlet environment. Some of our ExchangeFilterFunction requires current authentication object as well as values associated to the http request. Previously, with RestTemplate, we used ThreadLocal to hold and retrieve request bounded values.(RequestContextHolder, SecurityContextHolder, etc.)

With WebClient, Spring Security's SecurityReactorContextSubscriber in SecurityReactorContextConfiguration exposes auth, servlet request, and servlet response in a Map(called ContextAttributes) to the subscriber context.

In our custom filter functions, we leverage this context-attributes-map to retrieve auth and request bounded values similar to what ServletBearerExchangeFilterFunction does.

Currently, to access this context-attributes-map, the key SECURITY_CONTEXT_ATTRIBUTES is defined as package private in SecurityReactorContextConfiguration.

static final String SECURITY_CONTEXT_ATTRIBUTES = "org.springframework.security.SECURITY_CONTEXT_ATTRIBUTES";

Even within Spring Security, ServletBearerExchangeFilterFunction(oauth2-resource-server module) and ServletOAuth2AuthorizedClientExchangeFilterFunction(oauth2-client module) which define same String constant key in order to access this context-attributes-map. This is probably due to the SECURITY_CONTEXT_ATTRIBUTES constant is defined in SecurityReactorContextConfiguration(config module) which is a downstream module(and class is package private).

So, to allow context-attributes-map being used not only by oauth2-[resource-server|client] modules, but also user library/application, can this SECURITY_CONTEXT_ATTRIBUTES constant (and maybe mechanism around) be exposed to somewhere in upstream module public class? Also, the context-attributes-map can be an independent class instead of generic Map to enhance accessibility of stored objects.

Ideally, I think this infrastructure(subscriber, lifter, context-attributes-map) is available in Spring Framework; then, Spring Security adds a logic to populate SecurityContext on it. (https://github.com/spring-projects/spring-framework/issues/25710)

In servlet environment, when migrating RestTemplate to WebClient, one of the challenges people face is to access http request bound objects within WebClient filters(ExchangeFilterFunction). I think this is an area currently lacking a support.

For example, SecurityContext, Locale/LocaleContext(LocaleContextHolder), HttpServletRequest(RequestContextHolder) and any values associated to the request(ThreadLocal) cannot easily be retrieved in ExchangeFilterFunction.

I think common solution for this is to use Hooks to populate the values to subscriber's context.

For example, in Spring Security, here defines and registers a hook that populates SecurityContext. Since this mechanism also populates HttpServletRequest and HttpServletResponse, I leverage it in my exchange filter functions to retrieve request bound values. I also have similar logic for MDC and LocaleContext.

I think this mechanism should be supported in Spring Framework itself; so that, all downstream libraries and application can leverage it.

For implementation perspective, for example, define a class ReactorContextAttribute which is a map kept in thread local. Then, framework guarantees to populate this map in subscriber's context. So that, users or downstream libraries can simply populate this map in request thread, then retrieves values from the map in subscriber context. In addition, FrameworkServlet/DispatcherServlet resolved value/context, such as LocaleContext, can be placed in this map to make them accessible in exchange filter functions.

If such mechanism exists, for example, Spring Security can simply add a logic to populate SecurityContext.

I think this is a big missing piece to use WebClient in servlet environment along with supporting MDC.

@Bean javadoc states when BFPP is defined with non-static method, it writes a WARN-level log message.

This works out in BFPP cases, as they are not typically referenced by other @Bean methods. As a reminder, a WARN-level log message will be issued for any non-static @Bean methods having a return type assignable to BeanFactoryPostProcessor

The commit 9a43d2ec208d2e8cd0866431acf26af3529f8677 (SPR-16946, #21485) updated the logging level in ConfigurationClassEnhancer and this message shows up as INFO log.

if (logger.isInfoEnabled() &&
    BeanFactoryPostProcessor.class.isAssignableFrom(beanMethod.getReturnType())) {
  logger.info(String.format("@Bean method %s.%s is non-static and returns an object " +
          "assignable to Spring's BeanFactoryPostProcessor interface. This will " +
          "result in a failure to process annotations such as @Autowired, " +
          "@Resource and @PostConstruct within the method's declaring " +
          "@Configuration class. Add the 'static' modifier to this method to avoid " +
          "these container lifecycle issues; see @Bean javadoc for complete details.",
      beanMethod.getDeclaringClass().getSimpleName(), beanMethod.getName()));
}

So, either javadoc or log level need to be updated to match actual description and behavior.

One of the application team reported an issue and I found this slightly unintuitive behavior between condition evaluation of @ConditionalOn*Bean and spring's autowiring resolution for method argument.

Sample Code:

public class AnotherApplication {
	public static void main(String[] args) {
		SpringApplication.run(Config.class, "--debug");
	}

	@SpringBootConfiguration(proxyBeanMethods = false)
	static class Config {

		@Bean
		Parent foo() {
			return new Child("FOO");
		}

		@Bean
		Child bar() {
			return new Child("BAR");
		}

//		@Bean
//		Parent baz() {
//			return new Child("BAZ");
//		}

		@Bean
		@ConditionalOnSingleCandidate(Child.class)	// <<<--
		InitializingBean init(Child child) {		// <<<--
			return () -> {
				System.out.println("initialized");
			};
		}

	}

	static class Parent {
		String name;

		public Parent(String name) {
			this.name = name;
		}
	}

	static class Child extends Parent {
		public Child(String name) {
			super(name);
		}
	}
}

Condition evaluation report:

Positive matches:
-----------------

   AnotherApplication.Config#init matched:
      - @ConditionalOnSingleCandidate (types: com.example.AnotherApplication$Child; SearchStrategy: all) found a primary bean from beans 'bar' (OnBeanCondition)

Failure:

***************************
APPLICATION FAILED TO START
***************************

Description:

Parameter 0 of method init in com.example.AnotherApplication$Config required a single bean, but 2 were found:
	- foo: defined by method 'foo' in com.example.AnotherApplication$Config
	- bar: defined by method 'bar' in com.example.AnotherApplication$Config

Analysis

While performing OnBeanCondition(@ConditionalOnSingleCandidate), it compares the target type(Child) against factory-method-return-type of each @Bean - Parent for bean foo and Child for bar. Therefore, Child matches to bar bean and satisfies the condition.

The details of this process is, in OnBeanCondition, it calls beanFactory.getBeanNamesForType to collect bean names for the specified type(Child). In that, it calls AbstractBeanFactory#getSingleton. At this time, spring did not create raw beans yet, thus, the beanInstance becomes null. Then, it goes to check the bean definition's factoryMethodReturnType against specified condition type Child.

On the other hand, when spring performs autowiring for method arguments, it also calls beanFactory.getBeanNamesForType. At this time, raw beans are available, and it finds the actual bean instances. Then, it proceeds to perform instanceof check against the method argument type Child. This instanceof matches to both foo(Parent) and bar(Child) beans. Therefore, it detects two beans and fails to resolve which one to apply for the method argument.

There is another not intuitive behavior.

In following situation, it will not match the condition because the return type of foo is Parent.

@Configuration(proxyBeanMethods = false)
static class Config {

  @Bean
  Parent foo() {
    return new Child("FOO");
  }

  @Bean
  @ConditionalOnBean(Child.class)
  InitializingBean init(Child child) {
    ...
  }

}

Negative matches:
-----------------

   AnotherApplication.Config#init:
      Did not match:
         - @ConditionalOnBean (types: com.example.AnotherApplication$Child; SearchStrategy: all) did not find any beans of type com.example.AnotherApplication$Child (OnBeanCondition)

However, once I add Child bar() bean, suddenly the condition matches but autoconfiguration for Child fails with finding two beans - foo and bar.

@Configuration(proxyBeanMethods = false)
static class Config {

  @Bean
  Parent foo() {
    return new Child("FOO");
  }

  @Bean
  Child bar() {
    return new Child("BAR");
  }

  @Bean
  @ConditionalOnBean(Child.class)
  InitializingBean init(Child child) {
    ...
  }

}

Positive matches:
-----------------

   AnotherApplication.Config#init matched:
      - @ConditionalOnBean (types: com.example.AnotherApplication$Child; SearchStrategy: all) found bean 'bar' (OnBeanCondition)

***************************
APPLICATION FAILED TO START
***************************

Description:

Parameter 0 of method init in com.example.AnotherApplication$Config required a single bean, but 2 were found:
	- foo: defined by method 'foo' in com.example.AnotherApplication$Config
	- bar: defined by method 'bar' in com.example.AnotherApplication$Config

This suddenly matching multiple beans issue has reported in our application and I found this behavior.

I understand this is possibly working as designed.
The difference is due to the behavior of beanfactory.getBeanNamesForType which changes slightly different based on the stage this method is called. And also due to that bean instances cannot be checked while processing conditions.

It might be a corner case, but may be good to document such behavior on @ConditonalOn*Bean annotations.

FYI, If you want to log bindings and query all together(also connection id), r2dbc-proxy provides such capability. Here is sample log. (It is multilined for display purpose but actual logging is single line, and configurable to own format)

By definition, this method should not return null.

The usecase for me is more on testing side at the moment. When ConnectionFactory is created automatic(dynamic) fashion (e.g: spring-boot auto configuration), caller may not know how many times it is wrapped. So, need to introspect(unwrap) each of the object until it finds what it is looking for.

This, in turn, same for users who need to use implementation specific feature(one that is not available in SPI). The logic may not know how the object is constructed, so need to loop through the unwrapping.

The proposed API would not matter unwrapping type is either same as wrapping type or can be even different(not sure it's realistic or not though).

Currently Wrapped#unwrap retrieves the holding original object on the instance. If an object is wrapped multiple times, then the caller needs to iterate through the target to find the one that is looking for to be unwrapped.

For example, if an object ConnectionFactory is wrapped as proxy -> pool -> H2 ConnectionFactory, caller needs to unwrap twice to get the original H2 connection-factory.

Similar to Wrapper#unwrap(Class) in JDBC, it is helpful for users to directly return unwrapping target object if it is available from wrapped object, or returns null.

This can be implemented in default method on the Wrapped interface.(Or static method) So that, each implementation does not need to handle the call.

Here is a sample implementation with default method to find the first (outer most) wrapped object that matches to the given class.

@Nullable
@SuppressWarnings("unchecked")
default <E> E unwrapTo(Class<E> targetClass) {
    Assert.requireNonNull(targetClass, "targetClass must not be null");
    Object target = this;
    while (true) {
        if (targetClass.isInstance(target)) {
            return (E) target;
        }
        if (!(target instanceof Wrapped)) {
            return null;
        }
        target = ((Wrapped<?>) target).unwrap();
    }
}

We can have more generic one with Predicate:

@Nullable
default <E> E unwrapTo(Class<E> targetClass) {
    return unwrapTo(targetClass::isInstance);
}

@Nullable
@SuppressWarnings("unchecked")
default <E> E unwrapTo(Predicate<Object> predicate) {
    ....
}

For boolean check (if we want), we could reuse above like:

default boolean isWrapping(Class<?> targetClass) {
    return unwrapTo(targetClass) != null;
}

@snicoll While writing tests, I found using Option has a problem wrapping ConnectionPool with proxy. This is because @Bean method for ConnectionPool wraps the result of createConnectionFactory(). Therefore, proxy cannot wrap the ConnectionPool. So, I fallback to the original plan to make a hook on createConnectionFactory() to post process configured ConnectionFactory. (which I also applied to creating ConnectionPool)

I was working on readiness/liveness, so I was only looking them especially they are not enabled by default. But it is even better if all properties are documented on the table there. It is easy for readers to have information there how to enable healthindicators (though other than readiness/liveness are enabled by default). I don't think it is easy for readers to realize there is an appendices that have all properties listed for actuator unless there is an explicit link in the section.

@wilkinsona Thanks for the pointer.

The logging on the code above only happen when the actual health check logic throws an exception. Some of the healthcheck implementation throws Exception and let here to set DOWN status, but some are constructing response health status in the logic without throwing exception.

The intention for my case is when health check(aggregated) returns non UP state(DOWN, OUT_OF_SERVICE, UNKNOWN), then perform logging. This is because the response of liveness health group(/actuator/health/liveness) will trigger the restart of the application. When it happens, I would like to come back to see application log and find out which health indicator returned non-UP status as well as details if available.

Instead of referencing readinessStateProbeIndicator and livenessStateProbeIndicator, I think you need to set management.health.livenessstate.enabled and management.health.readinessstate.enabled properties introduced by spring-boot 2.3.2. So that, you could use readinessState and livenessState reference.

When management.health.[readiness|livenessstate].enabled properties are set to false(by default), AvailabilityProbesAutoConfiguration creates readinessStateProbeIndicator and livenessStateProbeIndicator beans which need to be referenced as [readiness|liveness]StateProbeIndicator(full bean name).

On the other hand, when properties are enabled, AvailabilityHealthContributorAutoConfiguration creates [readiness|liveness]StateHealthIndicator beans which can be referenced as [readiness|liveness]State.

The problem is in AvailabilityProbesHealthEndpointGroups created by AvailabilityProbesHealthEndpointGroupsPostProcessor, this creates readiness/liveness groups with [readiness|liveness]State. So, if [readiness|liveness]State are not available, groups are created but referenced HealthIndicator beans are not there.

This is similar to #22509 to improve root cause analysis when probe endpoints returned non 200 response.

I am migrating k8s http probes to use readiness and liveness health group endpoints(/actuator/health/[readiness|liveness]). When these endpoints return non UP status(other than 200 response), k8s stops traffic or shutdown the pod. When such event happens, k8s http probe only record the returned http status for the reason of its probe failure. This makes hard to investigate WHY readiness/liveness probes returned non 200 response when somebody needs to investigate the failure reason later. Even if k8s could record body of probe response, it would be nicer to have such information in application log.

I wrote this implementation to our services to log information when health endpoints returns non UP response.

@Slf4j
public class LoggingHealthEndpointWebExtension extends HealthEndpointWebExtension {

	public LoggingHealthEndpointWebExtension(HealthContributorRegistry registry, HealthEndpointGroups groups) {
		super(registry, groups);
	}

	@Override
	public WebEndpointResponse<HealthComponent> health(ApiVersion apiVersion, SecurityContext securityContext,
			boolean showAll, String... path) {
		WebEndpointResponse<HealthComponent> response = super.health(apiVersion, securityContext, showAll, path);
		HealthComponent health = response.getBody();
		if (health == null) {
			return response;
		}

		Status status = health.getStatus();
		if (status != Status.UP) {
			Map<String, HealthComponent> components = new TreeMap<>();
			if (health instanceof CompositeHealth) {
				Map<String, HealthComponent> details = ((CompositeHealth) health).getComponents();
				if (details != null) {
					components.putAll(details);
				}
			}
			log.warn("Health endpoints {} returned {}. components={}", path, status, components);
		}

		return response;
	}

}

If HealthEndpointSupport could have logging capability (or HealthEndpointWebExtension and ReactiveHealthEndpointWebExtension for web only), then we don't need to have this custom implementation.

Something like:

boolean enableLogging;

if(this.enableLogging && health.getStatus() != Status.UP) {
  log.warn(...);
}

Document property names to enable probe(readiness/liveness) health indicators.

<!-- Thanks for contributing to Spring Boot. Please review the following notes before submitting you pull request.

Please submit only genuine pull-requests. Do not use this repository as a GitHub playground.

Security Vulnerabilities

STOP! If your contribution fixes a security vulnerability, please do not submit it. Instead, please head over to https://pivotal.io/security to learn how to disclose a vulnerability responsibly.

Dependency Upgrades

Please do not open a pull request for a straightforward dependency upgrade (one that only updates the version property). We have a semi-automated process for such upgrades that we prefer to use. However, if the upgrade is more involved (such as requiring changes for removed or deprecated API) your pull request is most welcome.

Describing Your Changes

If, having reviewed the notes above, you're ready to submit your pull request, please provide a brief description of the proposed changes. If they fix a bug, please describe the broken behaviour and how the changes fix it. If they make an enhancement, please describe the new functionality and why you believe it's useful. If your pull request relates to any existing issues, please reference them by using the issue number prefixed with #. -->

Prior to this change, TaskExecutorBuilder used second as its precision to map awaitTerminationPeriod value. This caused a loss of millisecond information in the period. This commit fixes the loss by converting the period to millisecond and use the setAwaitTerminationMillis on executor/scheduler.

<!-- Thanks for contributing to Spring Boot. Please review the following notes before submitting you pull request.

Please submit only genuine pull-requests. Do not use this repository as a GitHub playground.

Security Vulnerabilities

STOP! If your contribution fixes a security vulnerability, please do not submit it. Instead, please head over to https://pivotal.io/security to learn how to disclose a vulnerability responsibly.

Dependency Upgrades

Please do not open a pull request for a straightforward dependency upgrade (one that only updates the version property). We have a semi-automated process for such upgrades that we prefer to use. However, if the upgrade is more involved (such as requiring changes for removed or deprecated API) your pull request is most welcome.

Describing Your Changes

If, having reviewed the notes above, you're ready to submit your pull request, please provide a brief description of the proposed changes. If they fix a bug, please describe the broken behaviour and how the changes fix it. If they make an enhancement, please describe the new functionality and why you believe it's useful. If your pull request relates to any existing issues, please reference them by using the issue number prefixed with #. -->

@ttddyy I can see a WIP in your last update and the tests have been commented out. Do you need something from us? Thanks a lot for your effort.

@snicoll Thanks for checking. I wanted to check whether the current impl with customizer is ok before I proceed. And since you said ok, I'll finish up the rest of the PR.

Changing availability state(readiness/liveness/etc.) is significant events for application. Currently, there is no logging around the availability state change and this makes it hard to find out what caused the state to change when somebody needs to look back what has happened to the application. So, we would like to capture the state change event in application log.

The caller(event publisher) side can log the event but it is more enforced if ApplicationAvailabilityBean(receiver side) logs the event.

For now, state change logging can be achieved by implementing a custom ApplicationListener like followings.

@Slf4j
@Order(0)  // make sure this listener run before "ApplicationAvailabilityBean"
class AvailabilityStateChangeLoggingListener implements ApplicationListener<AvailabilityChangeEvent<?>> {

  private final ApplicationAvailability availability;

  public AvailabilityStateChangeLoggingListener(ApplicationAvailability availability) {
    this.availability = availability;
  }

  @Override
  public void onApplicationEvent(AvailabilityChangeEvent<?> event) {
    AvailabilityState newState = event.getState();
    AvailabilityState lastState = this.availability.getState(newState.getClass());
    if (lastState != null && newState != lastState) {
      Class<? extends AvailabilityState> stateType = getStateType(newState);
      log.info("AvailabilityState changed. type={}, new={}, last={}, source={}",
          stateType.getSimpleName(), newState, lastState, event.getSource().getClass().getSimpleName());
    }
  }

  @SuppressWarnings("unchecked")
  private Class<? extends AvailabilityState> getStateType(AvailabilityState state) {
    if (state instanceof Enum) {
      return (Class<? extends AvailabilityState>) ((Enum<?>) state).getDeclaringClass();
    }
    return state.getClass();
  }

}

But the caveat is this needs to implement @Order/Ordered to ensure the listener ordering correct to retrieve the last state.

If ApplicationAvailabilityBean logs state change, this custom listener is not needed.

After discussing with @snicoll we found that ordering of pool/proxy/driver wrapping matters (speaking in URL syntax, r2dbc:pool:proxy:h2 vs. r2dbc:proxy:pool:h2) if someone wants to e.g. measure how many real connections were created/closed by the pool vs. how long a connection is in use by the application.

Yes, the ordering matters, and it can be even r2dbc:proxy:pool:proxy:h2.(though, there is a problem specifying listeners for which proxy to apply since there is only one proxyListener option) I consider most of the case proxy is the outer most wrapper since you want to check the Application behavior. If a user wants to proxy the middle ConnectionFactory, the user needs to wrap it manually, but I think that is more advanced use case. For connection-pool in Spring Boot, it may provide a flag in connection-pool settings which would be specific to proxying the connection-pool with some canned proxy listener such as metrics.

Currently, ConnectionFactoryOptionsBuilderCustomizer provides a way to customize Options on ConnectionFactoryBuilder. So, I could use it as a hook point to add/replace some of the Options.

I wasn't sure it is ok to use customizer facility within Spring Boot, but if that's ok, then here can define a customizer bean (probably with @Order to allow users to run own customizers before and after). Then, it will be pretty much all we need for r2dbc-proxy.

ok, I made not to throw NoClassDefFoundError but I'll go some other way not to use them here.

What here is doing is reconstructing DRIVER and PROTOCOL options.

e.g: r2dbc:h2:mem://... to r2dbc:proxy:h2:mem://...

This way, the discovery mechanism will create ConnectionFactory for proxy and mysql.

Actually, I think same way may be used for r2dbc-pool via PoolingConnectionFactoryProvider as well.

Hi @snicoll I added a commit 9cccc61004fa4616a02d32b3572149675f3f329b.

This uses ConnectionFactoryOptionsInitializer(via ConnectionFactoryConfigurations#createConnectionFactory) to construct ConnectionFactory options used by ProxyConnectionFactoryProvider(r2bc-proxy). Also, added ProxyListenersProvider interface to avoid direct reference to the classes in r2dbc-proxy to make it work when r2dbc-proxy is not available.

If this looks good, I'll cleanup the commits with tests.

@snicoll Ok, I'll explore the path for modifying ConnectionFactoryConfigurations#createConnectionFactory with ConnectionFactoryOptions. ProxyConnectionFactoryProvider(ConnectionFactoryProvider impl in r2dbc-proxy) understands proxy-listener options. So, if proxy-listener beans are defined, pass them as ConnectionFactoryOptions, then ConnectionFactories#find() would delegate to ProxyConnectionFactoryProvider for actual ProxyConnectionFactory creation with those proxy-listener beans. I think this way is better since it would make more use of r2dbc-spi infrastructure.

Hi @marcingrzejszczak

I wrote a sample listener for tracing here long time ago. Looking at it now, I can cleanup a bit. I'll find a time to update it as for a sample listener impl for tracing.