Elasticsearch to OpenSearch Migration - Prerequisites

OpenSearch

2022-05-25

Elasticsearch to OpenSearch Migration - Prerequisites

In this article series, we will guide you through migrating from Elasticsearch to OpenSearch step-by-step. We start with checking some prerequisites and the question: Should you migrate at all? Spoiler alert: Yes, you probably should.

Why migrate from Elasticsearch to OpenSearch?

The first and probably most important question is: Why should you consider migrating from Elasticsearch to OpenSearch at all? The short answer is - to avoid vendor lock-in and secure your IT investment.
Elasticsearch started as an Open Source, Apache2-licensed project which anyone could use free of charge and for any purpose. As with many Open Source business models, Elastic only charges for feature add-ons, like Alerting, Machine Learning and also basic security features. But the core Elasticsearch platform was always ASL2-licensed.
Although Elastic (the company behind Elasticsearch) promised there would always be an Open Source version of Elasticsearch, this changed in 2021 and new license conditions were applied. Elasticsearch is not an Open Source project anymore, and no one knows what the future will bring regarding potential new license changes and/or changes in the Elastic business model.
With OpenSearch, people benefit from having an Open Source product they can use, modify, extend, monetize, and resell how they want.
OpenSearch is a fork of Elasticsearch licensed under Apache2. It is a community-driven project, backed by industry leaders such as AWS, RedHat, SAP, Logz.io and more. As the project puts it:
“With OpenSearch, people benefit from having an Open Source product they can use, modify, extend, monetize, and resell how they want. At the same time, OpenSearch will continue to provide a secure, high-quality search and analytics suite with a rich roadmap of new and innovative functionality.” Read more here.

Benefits and Potential Drawbacks

As always, changing platforms has potential benefits and drawbacks. Since OpenSearch is a direct fork of Elasticsearch, compatibility is high, and the benefits will most probably outweigh the drawbacks.

Benefits

    Free (as in free beer) and ASL2 licensed - use it in any way you want without worrying about license changes
    Community-driven and backed by major industry leaders like AWS, RedHat and SAP
    You can offer managed services based on OpenSearch, which is not possible with Elasticsearch
    Steady contributions and improvements by the community
    You can contribute to the project yourself, and your code will stay under the ASL2 license
    Project roadmap is publicly visible which provides planning security
    Bug fixes and security fixes will be backported continuously

Drawbacks

Feature Comparison

The following table compares the (in our experience) most used features of Elasticsearch with OpenSearch. Before planning your migration, check if all features you use are also available on OpenSearch, or if an alternative or workaround is available. See also this post on the AWS blog.

Basic Features

Feature Elasticsearch OpenSearch
Basic Security Features yes yes
Advanced Security Features yes, paid yes
Audit Logging yes, paid yes
Alerting yes, paid yes
Machine Learning yes, paid yes
Anomaly Detection yes, paid yes
Index Lifecycle Management yes, paid yes
SQL Interface yes, paid yes
Log Analytics yes yes
Reporting yes yes
Cross-cluster search yes yes
Cross-cluster replication yes, paid yes

Special Features

Feature Elasticsearch OpenSearch
Full stack monitoring yes, paid not yet
Fleet yes, paid no
APM yes, paid no
Data Streams no yes
Transforms no yes
Dashboard Notebooks no yes
Endpoint Security yes no

Data Compatibility

When migrating from Elasticsearch to OpenSearch, you can re-use your existing index data. Since OpenSearch is a fork of Elasticsearch 7.10.2, and since Lucene powers both under the hood anyway, you can use the data directories created by Elasticsearch also with OpenSearch.
If you are running older versions of Elasticsearch, we recommend upgrading to Elasticsearch 7.10.2 before migrating to OpenSearch.
If you are running newer versions of Elasticsearch, especially if you have already upgraded to Elasticsearch 8, please check the Elasticsearch release notes to see if there are any breaking changes regarding the data format on disk. In this case, we recommend using the snapshot approach to migrate your data.

Configuration Compatibility

The configuration of Elasticsearch and OpenSearch is same-same, but also different. Depending on the Elasticsearch features you use, you will need to manually adapt configuration files like elasticsearch.yml or users and security roles to conform with the OpenSearch format.
There’s also an opensearch-upgrade tool which you can use to automate some of these changes.

Upgrade Path: Three modes

There are three major upgrade paths you can follow (from safe to moderate risk):

Snapshot/Restore approach

In this scenario, you set up a new OpenSearch cluster. You export the data from your running Elasticsearch cluster and import it to your OpenSearch cluster.

Full Cluster Restart

In this scenario, you take down your existing Elasticsearch cluster completely. You then spin up your new OpenSearch cluster, pointing it to the data directories of the former Elasticsearch cluster.

Rolling Restart

In this scenario, you upgrade your cluster node-by-node. You take down one node, upgrade it to OpenSearch, and add it back to the cluster. This approach ensures there is no service interruption, but it is also the riskiest.
In the next parts of this series, we will look at the different approaches, discuss their pros and cons, and provide some real-world and hands-on examples. Stay tuned!
In the meantime, you can also check out or Elasticsearch to OpenSearch Migration FAQ.
Get in touch with us
Send us a message and talk to the team today.
Need help with your setup?
OpenSearch or Elastic Stack – We’ve got you covered.