Skip to content

Client hardcodes Accept header version with no override, breaking compatibility with older Elasticsearch servers #8526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
J-ohn opened this issue May 15, 2025 · 3 comments

Comments

@J-ohn
Copy link

J-ohn commented May 15, 2025

Elastic.Clients.Elasticsearch version:
9.x (e.g., 9.1.0)

Elasticsearch version:
8.x (e.g., 8.11.3) — varies across deployments

.NET runtime version:
.NET 8.0 (also applies to .NET 6.0 and 7.0)

Operating system version:
Cross-platform / environment-agnostic


Description of the problem including expected versus actual behavior:
The Elastic.Clients.Elasticsearch v9 client hardcodes the Accept header to:

application/vnd.elasticsearch+json;compatible-with=9

This is not configurable via public API.

Our product is deployed into a wide range of external environments, each of which is responsible for managing its own Elasticsearch instance. These instances are often running older versions (e.g., Elasticsearch 8.x), and the teams managing them are outside of our control. We must maintain compatibility across all these targets, provided the feature set being used is supported by the target server.

Because the client forces the compatible-with=9 header, it fails when connecting to an Elasticsearch 8.x server—even though the actual operation is otherwise compatible. There is no way to opt out of or override this header short of reflection.

In addition, when using Elastic Cloud, the server may be upgraded automatically or by a different team without coordination. This creates a situation where the server changes out from under the client, and version rigidity in the header causes preventable runtime failures.

This tight coupling of client version to server version via HTTP headers introduces brittleness, prevents safe incremental upgrades, and creates unnecessary operational risk.


Steps to reproduce:

  1. Use Elastic.Clients.Elasticsearch v9.x in a .NET application.
  2. Connect to an Elasticsearch 8.x server (e.g., in Elastic Cloud or any managed environment).
  3. Perform a standard operation (e.g., SearchAsync).
  4. Observe the error caused by the incompatible Accept header.

Expected behavior
The client should:

  • Allow overriding the Accept header or compatibility version via a public configuration API.
  • Or, gracefully negotiate compatibility with the server when feasible.

This would enable us to upgrade the client (e.g., for bug or security fixes) without forcing downstream consumers to upgrade their Elasticsearch servers, especially in environments we don’t control.


Provide ConnectionSettings (if relevant):

var settings = new ElasticsearchClientSettings(CreatePool(),
	sourceSerializer: (builtin, config) =>
		new ElasticsearchJsonNetSerializer(serviceProvider.GetRequiredService<JsonSerializerSettingsFactory>())
);

// Workaround using reflection (no public API available)
var field = typeof(TransportConfigurationDescriptorBase<ElasticsearchClientSettings>)
	.GetField("_accept", BindingFlags.Instance | BindingFlags.NonPublic);

field?.SetValue(settings, "application/vnd.elasticsearch+json;compatible-with=8");

Provide DebugInformation (if relevant):
Error from server:

Content-Type header [application/vnd.elasticsearch+json;compatible-with=9] is not supported
@J-ohn J-ohn added Category: Bug 9.x Relates to a 9.x client version labels May 15, 2025
@flobernd
Copy link
Member

Hi @J-ohn ,

this is by design.

Please have a look at the "Compatibility Matrix" section in the README:

Language clients are forward compatible; meaning that the clients support communicating with greater or equal minor versions of Elasticsearch without breaking.

The 9.x client is only compatible with 9.x and 10.x servers.

In addition, when using Elastic Cloud, the server may be upgraded automatically or by a different team without coordination. This creates a situation where the server changes out from under the client, and version rigidity in the header causes preventable runtime failures.

This tight coupling of client version to server version via HTTP headers introduces brittleness, prevents safe incremental upgrades, and creates unnecessary operational risk.

I understand that the server environment might be out of your control, but the official recommended way for upgrades is "server first" for a reason...

The whole purpose of this header is to make sure that the client and the server is actually compatible. Disabling/modifying it defeats its purpose and contrary to your statement will be an actual source of failures.

If an ES 9.x server sees the compatible-with=8 header, it will switch to a 8.x compatibility mode that actually guarantees full compatibility with the 8.x client.

An ES 8.x server obviously does not know about the capabilities of a 9.x client, changed, removed, added fields etc. which means that we can never guarantee that a 9.x client is compatible with a 8.x server.


Luckily there is a very simple solution to your problem, that allows you to be compatibly with 8.x and 9.x Elasticsearch clusters using the exact same code base:

➡ Use the latest 8.x client.


If you want to use the 9.x client and you don't care about potential runtime failures, you can override Accept and Content-Type on a per-request basis like this to disable the versioning header:

await client.IndexAsync(new Person(), x => x.RequestConfiguration(x => x
    .Accept("application/json")
    .ContentType("application/json"))
);

Please note that this is completely unsupported and we won't provide any support for this use-case.

Please let me know, if that answers your question / solves your issue.

Copy link
Contributor

This issue is stale because it has been open 5 days with no activity. Remove stale label or comment or this will be closed in 2 days.

Copy link
Contributor

This issue was closed because it has been stalled for 2 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants