Use the new Rest5Client as default, provide the old RestClient as optional.

pom.xml

@@ -132,6 +132,7 @@
 			</exclusions>
 		</dependency>
 
+		<!-- the old RestCLient is an optional dependency for user that still want to use it-->
 		<dependency>
 			<groupId>org.elasticsearch.client</groupId>
 			<artifactId>elasticsearch-rest-client</artifactId>
@@ -142,6 +143,7 @@
 					<artifactId>commons-logging</artifactId>
 				</exclusion>
 			</exclusions>
+			<optional>true</optional>
 		</dependency>
 
 		<dependency>

src/main/antora/modules/ROOT/pages/elasticsearch/clients.adoc

@@ -6,10 +6,10 @@ This chapter illustrates configuration and usage of supported Elasticsearch clie
 Spring Data Elasticsearch operates upon an Elasticsearch client (provided by Elasticsearch client libraries) that is connected to a single Elasticsearch node or a cluster.
 Although the Elasticsearch Client can be used directly to work with the cluster, applications using Spring Data Elasticsearch normally use the higher level abstractions of xref:elasticsearch/template.adoc[Elasticsearch Operations] and xref:elasticsearch/repositories/elasticsearch-repositories.adoc[Elasticsearch Repositories].
 
-[[elasticsearch.clients.restclient]]
-== Imperative Rest Client
+[[elasticsearch.clients.rest5client]]
+== Imperative Rest5Client
 
-To use the imperative (non-reactive) client, a configuration bean must be configured like this:
+To use the imperative (non-reactive) Rest5Client, a configuration bean must be configured like this:
 
 ====
 [source,java]
@@ -31,7 +31,7 @@ public class MyClientConfig extends ElasticsearchConfiguration {
 <.> for a detailed description of the builder methods see xref:elasticsearch/clients.adoc#elasticsearch.clients.configuration[Client Configuration]
 ====
 
-The javadoc:org.springframework.data.elasticsearch.client.elc.ElasticsearchConfiguration[]] class allows further configuration by overriding for example the `jsonpMapper()` or `transportOptions()` methods.
+The javadoc:org.springframework.data.elasticsearch.client.elc.ElasticsearchConfiguration[] class allows further configuration by overriding for example the `jsonpMapper()` or `transportOptions()` methods.
 
 
 The following beans can then be injected in other Spring components:
@@ -46,7 +46,81 @@ ElasticsearchOperations operations;      <.>
 ElasticsearchClient elasticsearchClient; <.>
 
 @Autowired
-RestClient restClient;                   <.>
+Rest5Client rest5Client;                 <.>
+
+@Autowired
+JsonpMapper jsonpMapper;                 <.>
+----
+
+<.> an implementation of javadoc:org.springframework.data.elasticsearch.core.ElasticsearchOperations[]
+<.> the `co.elastic.clients.elasticsearch.ElasticsearchClient` that is used.
+<.> the low level `Rest5Client` from the Elasticsearch libraries
+<.> the `JsonpMapper` user by the Elasticsearch `Transport`
+====
+
+Basically one should just use the javadoc:org.springframework.data.elasticsearch.core.ElasticsearchOperations[] to interact with the Elasticsearch cluster.
+When using repositories, this instance is used under the hood as well.
+
+[[elasticsearch.clients.restclient]]
+== Deprecated Imperative RestClient
+
+To use the imperative (non-reactive) RestClient - deprecated since version 6 - , the following dependency needs to be added, adapt the correct version. The exclusion is needed in a Spring Boot application:
+====
+[source,xml]
+----
+        <dependency>
+            <groupId>org.elasticsearch.client</groupId>
+            <artifactId>elasticsearch-rest-client</artifactId>
+            <version>${elasticsearch-client.version}</version>
+            <exclusions>
+                <exclusion>
+                    <groupId>commons-logging</groupId>
+                    <artifactId>commons-logging</artifactId>
+                </exclusion>
+            </exclusions>
+        </dependency>
+
+----
+====
+
+The configuration bean must be configured like this:
+
+====
+[source,java]
+----
+import org.springframework.data.elasticsearch.client.elc.ElasticsearchLegacyRestClientConfiguration;
+
+@Configuration
+public class MyClientConfig extends ElasticsearchLegacyRestClientConfiguration {
+
+	@Override
+	public ClientConfiguration clientConfiguration() {
+		return ClientConfiguration.builder()           <.>
+			.connectedTo("localhost:9200")
+			.build();
+	}
+}
+----
+
+<.> for a detailed description of the builder methods see xref:elasticsearch/clients.adoc#elasticsearch.clients.configuration[Client Configuration]
+====
+
+The javadoc:org.springframework.data.elasticsearch.client.elc.ElasticsearchConfiguration[] class allows further configuration by overriding for example the `jsonpMapper()` or `transportOptions()` methods.
+
+
+The following beans can then be injected in other Spring components:
+
+====
+[source,java]
+----
+import org.springframework.beans.factory.annotation.Autowired;@Autowired
+ElasticsearchOperations operations;      <.>
+
+@Autowired
+ElasticsearchClient elasticsearchClient; <.>
+
+@Autowired
+RestClient restClient;                 <.>
 
 @Autowired
 JsonpMapper jsonpMapper;                 <.>
@@ -61,8 +135,8 @@ JsonpMapper jsonpMapper;                 <.>
 Basically one should just use the javadoc:org.springframework.data.elasticsearch.core.ElasticsearchOperations[] to interact with the Elasticsearch cluster.
 When using repositories, this instance is used under the hood as well.
 
-[[elasticsearch.clients.reactiverestclient]]
-== Reactive Rest Client
+[[elasticsearch.clients.reactiverest5client]]
+== Reactive Rest5Client
 
 When working with the reactive stack, the configuration must be derived from a different class:
 
@@ -99,6 +173,65 @@ ReactiveElasticsearchOperations operations;      <.>
 @Autowired
 ReactiveElasticsearchClient elasticsearchClient; <.>
 
+@Autowired
+Rest5Client rest5Client;                           <.>
+
+@Autowired
+JsonpMapper jsonpMapper;                         <.>
+----
+
+the following can be injected:
+
+<.> an implementation of javadoc:org.springframework.data.elasticsearch.core.ReactiveElasticsearchOperations[]
+<.> the `org.springframework.data.elasticsearch.client.elc.ReactiveElasticsearchClient` that is used.
+This is a reactive implementation based on the Elasticsearch client implementation.
+<.> the low level `RestClient` from the Elasticsearch libraries
+<.> the `JsonpMapper` user by the Elasticsearch `Transport`
+====
+
+Basically one should just use the javadoc:org.springframework.data.elasticsearch.core.ReactiveElasticsearchOperations[] to interact with the Elasticsearch cluster.
+When using repositories, this instance is used under the hood as well.
+
+[[elasticsearch.clients.reactiverestclient]]
+== Deprecated Reactive RestClient
+
+See the section above for the imperative code to use the deprecated RestClient for the necessary dependencies to include.
+
+When working with the reactive stack, the configuration must be derived from a different class:
+
+====
+[source,java]
+----
+import org.springframework.data.elasticsearch.client.elc.ReactiveElasticsearchLegacyRestClientConfiguration;
+
+@Configuration
+public class MyClientConfig extends ReactiveElasticsearchLegacyRestClientConfiguration {
+
+	@Override
+	public ClientConfiguration clientConfiguration() {
+		return ClientConfiguration.builder()           <.>
+			.connectedTo("localhost:9200")
+			.build();
+	}
+}
+----
+
+<.> for a detailed description of the builder methods see xref:elasticsearch/clients.adoc#elasticsearch.clients.configuration[Client Configuration]
+====
+
+The javadoc:org.springframework.data.elasticsearch.client.elc.ReactiveElasticsearchConfiguration[] class allows further configuration by overriding for example the `jsonpMapper()` or `transportOptions()` methods.
+
+The following beans can then be injected in other Spring components:
+
+====
+[source,java]
+----
+@Autowired
+ReactiveElasticsearchOperations operations;      <.>
+
+@Autowired
+ReactiveElasticsearchClient elasticsearchClient; <.>
+
 @Autowired
 RestClient restClient;                           <.>
 
@@ -183,8 +316,25 @@ In the case this is not enough, the user can add callback functions by using the
 
 The following callbacks are provided:
 
+[[elasticsearch.clients.configuration.callbacks.rest5]]
+==== Configuration of the low level Elasticsearch `Rest5Client`:
+
+This callback provides a `org.elasticsearch.client.RestClientBuilder` that can be used to configure the Elasticsearch
+`RestClient`:
+====
+[source,java]
+----
+ClientConfiguration.builder()
+    .connectedTo("localhost:9200", "localhost:9291")
+    .withClientConfigurer(Rest5Clients.ElasticsearchRest5ClientConfigurationCallback.from(restClientBuilder -> {
+        // configure the Elasticsearch Rest5Client
+        return restClientBuilder;
+    }))
+    .build();
+----
+====
 [[elasticsearch.clients.configuration.callbacks.rest]]
-==== Configuration of the low level Elasticsearch `RestClient`:
+==== Configuration of the deprecated low level Elasticsearch `RestClient`:
 
 This callback provides a `org.elasticsearch.client.RestClientBuilder` that can be used to configure the Elasticsearch
 `RestClient`:
@@ -193,26 +343,45 @@ This callback provides a `org.elasticsearch.client.RestClientBuilder` that can b
 ----
 ClientConfiguration.builder()
     .connectedTo("localhost:9200", "localhost:9291")
-    .withClientConfigurer(ElasticsearchClients.ElasticsearchRestClientConfigurationCallback.from(restClientBuilder -> {
+    .withClientConfigurer(RestClients.ElasticsearchRestClientConfigurationCallback.from(restClientBuilder -> {
         // configure the Elasticsearch RestClient
         return restClientBuilder;
     }))
     .build();
 ----
 ====
 
+[[elasticsearch.clients.configurationcallbacks.httpasync5]]
+==== Configuration of the HttpAsyncClient used by the low level Elasticsearch `Rest5Client`:
+
+This callback provides a `org.apache.hc.client5.http.impl.async.HttpAsyncClientBuilder` to configure the HttpClient that is
+used by the `Rest5Client`.
+
+====
+[source,java]
+----
+ClientConfiguration.builder()
+    .connectedTo("localhost:9200", "localhost:9291")
+    .withClientConfigurer(Rest5Clients.ElasticsearchHttpClientConfigurationCallback.from(httpAsyncClientBuilder -> {
+        // configure the HttpAsyncClient
+        return httpAsyncClientBuilder;
+    }))
+    .build();
+----
+====
+
 [[elasticsearch.clients.configurationcallbacks.httpasync]]
-==== Configuration of the HttpAsyncClient used by the low level Elasticsearch `RestClient`:
+==== Configuration of the HttpAsyncClient used by the deprecated low level Elasticsearch `RestClient`:
 
-This callback provides a `org.apache.http.impl.nio.client.HttpAsyncClientBuilder` to configure the HttpCLient that is
+This callback provides a `org.apache.http.impl.nio.client.HttpAsyncClientBuilder` to configure the HttpClient that is
 used by the `RestClient`.
 
 ====
 [source,java]
 ----
 ClientConfiguration.builder()
     .connectedTo("localhost:9200", "localhost:9291")
-    .withClientConfigurer(ElasticsearchClients.ElasticsearchHttpClientConfigurationCallback.from(httpAsyncClientBuilder -> {
+    .withClientConfigurer(RestClients.ElasticsearchHttpClientConfigurationCallback.from(httpAsyncClientBuilder -> {
         // configure the HttpAsyncClient
         return httpAsyncClientBuilder;
     }))

src/main/antora/modules/ROOT/pages/elasticsearch/elasticsearch-new.adoc

@@ -7,6 +7,7 @@
 * Upgarde to Spring 7
 * Switch to jspecify nullability annotations
 * Upgrade to Elasticsearch 9.0.3
+* Use the new Elasticsearch Rest5Client as default
 
 
 [[new-features.5-5-0]]

src/main/antora/modules/ROOT/pages/migration-guides/migration-guide-5.5-6.0.adoc

@@ -6,9 +6,13 @@ This section describes breaking changes from version 5.5.x to 6.0.x and how remo
 [[elasticsearch-migration-guide-5.5-6.0.breaking-changes]]
 == Breaking Changes
 
+From version 6.0 on, Spring Data Elasticsearch uses the Elasticsearch 9 libraries and as default the new `Rest5Client` provided by these libraries. It is still possible to use the old `RestClient`, check xref:elasticsearch/clients.adoc[Elasticsearch clients] for information. The configuration callbacks for this `RestClient` have been moved from `org.springframework.data.elasticsearch.client.elc.ElasticsearchClients` to the `org.springframework.data.elasticsearch.client.elc.rest_client.RestClients` class.
+
 [[elasticsearch-migration-guide-5.5-6.0.deprecations]]
 == Deprecations
 
+All the code using the old `RestClient` has been moved to the `org.springframework.data.elasticsearch.client.elc.rest_client` package and has been deprecated. Users should switch to the classes from the `org.springframework.data.elasticsearch.client.elc.rest5_client` package.
+
 
 === Removals
 

src/main/java/org/springframework/data/elasticsearch/client/ClientConfiguration.java

@@ -127,10 +127,16 @@ static ClientConfiguration create(InetSocketAddress socketAddress) {
 	Optional<String> getCaFingerprint();
 
 	/**
-	 * Returns the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if not configured.
+	 * Returns the {@link HostnameVerifier} to use. Must be {@link Optional#empty()} if not configured.
+	 * Cannot be used with the Rest5Client used from Elasticsearch 9 on as the underlying Apache http components 5 does not offer a way
+	 * to set this. Users that need a hostname verifier must integrate this in a SSLContext.
+	 * Returning a value here is ignored in this case
 	 *
 	 * @return the {@link HostnameVerifier} to use. Can be {@link Optional#empty()} if not configured.
+	 * @deprecated since 6.0
 	 */
+	// todo #3117 document this
+	@Deprecated(since = "6.0", forRemoval=true)
 	Optional<HostnameVerifier> getHostNameVerifier();
 
 	/**