What is OpenSearch?
What is OpenSearch and OpenSearch Dashboards and how you can benefit from this Open Source data analytics platform
Updated November 2022
OpenSearch is a highly scalable open-source search and analytics platform for a broad range of use cases. Data can be ingested from various sources and quickly searched, analyzed, and visualized. Prominent use cases include log analytics, application monitoring, anomaly detection, and website search.
OpenSearch and OpenSearch Dashboards were originally derived from Elasticsearch 7.10.2 and Kibana 7.10.2. AWS started the project at the beginning of this year. The goal is to make OpenSearch a genuine community-driven effort with several different maintainers. It consists of two main applications plus a set of plugins that further extend the capabilities of OpenSearch.
OpenSearch
Also called the “engine,” OpenSearch is a distributed, highly scalable, and highly available search and analytics engine at its core. You can ingest any data in JSON format and use the advanced search, analytics, and aggregation capabilities to make use of your data.
Your data is stored in a cluster and distributed amongst the participating nodes in this cluster. Nodes can be added and removed dynamically, and OpenSearch automatically balances and distributes the data amongst the nodes. An OpenSearch cluster can scale up to petabytes of data while providing ultra-low latency when searching or aggregating data.
While the most prominent use case is probably logs analytics, the possibilities are limitless. For example:
Storing and analyzing financial transactions and scanning them for anomalies to detect possible fraud.
Monitoring application performance in real-time.
Storing medical information like patient data or health-related data points from trackers, smartwatches, scales, and other devices.
Implementing a highly accurate and super fast application search.
OpenSearch runs on all major platforms, including Docker and Kubernetes.
OpenSearch Dashboards
OpenSearch Dashboard is the data visualization and management tool for OpenSearch. Use it to create stunning visualizations and dashboards of your data stored in Elasticsearch. Dashboards and visualizations can be shared and also embedded in other websites.
Dashboards also come with management and administrative features that you can use to monitor and configure OpenSearch.
OpenSearch Plugins
OpenSearch provides a plugin infrastructure so that anyone can extend the already rich capabilities of OpenSearch with new features. Both the OpenSearch engine and Dashboards support plugins.
AWS has already released a lot of powerful plugins, such as:
Security: Originally based on Search Guard, this plugin adds highly advanced and battle-proven role-based security controls so you can define what data a user is allowed to see. This includes index-, document- and field-level controls and an audit log for tracking and storing activities in your cluster for meeting compliance regulations.
Alerting: Get notified when data stored in OpenSearch meets specific criteria. For example, receive an email or slack message if the rate of errors on your application logs increased by 20% in the last 30 minutes.
Machine Learning: Use machine learning capabilities to automatically detect anomalies in your data or classify data automatically.
SQL: Instead of using the OpenSearch query DSL, you can also use plain-old SQL to query and aggregate data in OpenSearch. No need to learn a new language.
As OpenSearch, all plugins are licensed under Apache2.
OpenSearch is Vendor- and Cloud-agnostic
Although AWS initiated OpenSearch, it is by no means an “AWS only” project. OpenSearch is a community endeavor, and other companies have already joined in as well:
“We are not alone in our commitment to OpenSearch. Organizations as diverse as Red Hat, SAP, Capital One, and Logz.io have joined us in support.”
(AWS: Introdcing OpenSearch)
“Organizations as diverse as Red Hat, SAP, Capital One, and Logz.io have joined us in support.”
All code is published under the Apache2 license. This genuine open-source license allows you to download, run, modify, and distribute OpenSearch any way you want without consulting legal first.
OpenSearch is not limited to AWS or AWS cloud services. You can run it on any cloud platform of your choice or on-premise.
The Future of OpenSearch: Our Thoughts
We at Eliatra are very excited about OpenSearch for various reasons.
First, this is a true community-driven Open Source project that will always be licensed under Apache2. Elastic announced a couple of license changes this year, moving away from Open Source software altogether. This caused a lot of confusion among users, especially among companies that offered Elasticsearch as a managed service.
Constant changes in the licensing model put your investment at risk.
For some companies, the Elastic license changes broke their business model altogether. There is also uncertainty in the market regarding potential future modifications of the Elastic license model. OpenSearch is a better choice when it comes to safety regarding your investment in big data analytics.
AWS promised that while they will probably remain the steward of the OpenSearch project, it is open for anyone to participate and contribute. With big players like Red Hat, SAP, IBM or logz.io already contributing, we will see a thriving community and ecosystem in the future. By using OpenSearch, you can avoid the dreaded vendor lock-in. No more surprises when “circumstances have changed”.
Should You Migrate Now?
Update November 2022: OpenSearch is production-ready, and many companies have already migrated or are in the progress of migrating from Elasticsearch to OpenSearch.
Should You Plan to Migrate Now?
Absolutely yes. For all the reasons listed above (and more), you should look into OpenSearch and start your migration right now. If you need support or professional services to migrate from Elasticsearch to OpenSearch, Eliatra has it all:
Ready to get started?!
Let's work together to navigate your OpenSearch journey. Send us a message and talk to the team today!
Get in touch