OpenSearch

2022-05-25

Elasticsearch to OpenSearch Migration: Prerequisites

In this article series, we will guide you through migrating from Elasticsearch to OpenSearch step-by-step. We start with checking some prerequisites and the question: Should you migrate at all? Spoiler alert: Yes, you probably should.

Why migrate from Elasticsearch to OpenSearch?

The first and probably most important question is: Why should you consider migrating from Elasticsearch to OpenSearch at all? The short answer is - to avoid vendor lock-in and secure your IT investment.

Elasticsearch started as an Open Source, Apache2-licensed project which anyone could use free of charge and for any purpose. As with many Open Source business models, Elastic only charges for feature add-ons, like Alerting, Machine Learning and also basic security features. But the core Elasticsearch platform was always ASL2-licensed.

Although Elastic (the company behind Elasticsearch) promised there would always be an Open Source version of Elasticsearch, this changed in 2021 and new license conditions were applied. Elasticsearch is not an Open Source project anymore, and no one knows what the future will bring regarding potential new license changes and/or changes in the Elastic business model.

With OpenSearch, people benefit from having an Open Source product they can use, modify, extend, monetize, and resell how they want.

OpenSearch is a fork of Elasticsearch licensed under Apache2. It is a community-driven project, backed by industry leaders such as AWS, RedHat, SAP, Logz.io and more. As the project puts it:

“With OpenSearch, people benefit from having an Open Source product they can use, modify, extend, monetize, and resell how they want. At the same time, OpenSearch will continue to provide a secure, high-quality search and analytics suite with a rich roadmap of new and innovative functionality.” Read more here.

Benefits and Potential Drawbacks

As always, changing platforms has potential benefits and drawbacks. Since OpenSearch is a direct fork of Elasticsearch, compatibility is high, and the benefits will most probably outweigh the drawbacks.

Benefits

Free (as in free beer) and ASL2 licensed - use it in any way you want without worrying about license changes

Community-driven and backed by major industry leaders like AWS, RedHat and SAP

You can offer managed services based on OpenSearch, which is not possible with Elasticsearch

Steady contributions and improvements by the community

You can contribute to the project yourself, and your code will stay under the ASL2 license

Project roadmap is publicly visible which provides planning security

Bug fixes and security fixes will be backported continuously

Drawbacks

If you are using very specific Elasticsearch features, they might not be available (yet) on OpenSearch (see table below)

No official support from AWS available. However, companies like Eliatra (wink, wink) offer: OpenSearch support with guaranteed SLAs and custom development.

Feature Comparison

The following table compares the (in our experience) most used features of Elasticsearch with OpenSearch. Before planning your migration, check if all features you use are also available on OpenSearch, or if an alternative or workaround is available. See also this post on the AWS blog.

Basic Features

Feature	Elasticsearch	OpenSearch
Basic Security Features	yes	yes
Advanced Security Features	yes, paid	yes
Audit Logging	yes, paid	yes
Alerting	yes, paid	yes
Machine Learning	yes, paid	yes
Anomaly Detection	yes, paid	yes
Index Lifecycle Management	yes, paid	yes
SQL Interface	yes, paid	yes
Log Analytics	yes	yes
Reporting	yes	yes
Cross-cluster search	yes	yes
Cross-cluster replication	yes, paid	yes

Special Features

Feature	Elasticsearch	OpenSearch
Full stack monitoring	yes, paid	not yet
Fleet	yes, paid	no
APM	yes, paid	no
Data Streams	no	yes
Transforms	no	yes
Dashboard Notebooks	no	yes
Endpoint Security	yes	no

Data Compatibility

When migrating from Elasticsearch to OpenSearch, you can re-use your existing index data. Since OpenSearch is a fork of Elasticsearch 7.10.2, and since Lucene powers both under the hood anyway, you can use the data directories created by Elasticsearch also with OpenSearch.

If you are running older versions of Elasticsearch, we recommend upgrading to Elasticsearch 7.10.2 before migrating to OpenSearch.

If you are running newer versions of Elasticsearch, especially if you have already upgraded to Elasticsearch 8, please check the Elasticsearch release notes to see if there are any breaking changes regarding the data format on disk. In this case, we recommend using the snapshot approach to migrate your data.

Configuration Compatibility

The configuration of Elasticsearch and OpenSearch is same-same, but also different. Depending on the Elasticsearch features you use, you will need to manually adapt configuration files like elasticsearch.yml or users and security roles to conform with the OpenSearch format.

There’s also an opensearch-upgrade tool which you can use to automate some of these changes.

Upgrade Path: Three modes

There are three major upgrade paths you can follow (from safe to moderate risk):

Snapshot/Restore approach

In this scenario, you set up a new OpenSearch cluster. You export the data from your running Elasticsearch cluster and import it to your OpenSearch cluster.

Full Cluster Restart

In this scenario, you take down your existing Elasticsearch cluster completely. You then spin up your new OpenSearch cluster, pointing it to the data directories of the former Elasticsearch cluster.

Rolling Restart

In this scenario, you upgrade your cluster node-by-node. You take down one node, upgrade it to OpenSearch, and add it back to the cluster. This approach ensures there is no service interruption, but it is also the riskiest.

In the next parts of this series, we will look at the different approaches, discuss their pros and cons, and provide some real-world and hands-on examples. Stay tuned!

In the meantime, you can also check out or Elasticsearch to OpenSearch Migration FAQ.