From X-Pack to OpenSearch: Part 1 - TLS Setup
This article describes how to migrate TLS Settings from X-Pack Security to OpenSearch Security.
Reading time: 7 minutes
Summary: This article provides a comprehensive guide on migrating from X-Pack Security to OpenSearch Security, focusing on the similarities and differences in TLS encryption implementation between the two platforms. It details the process of identifying and configuring node certificates, generating admin certificates for OpenSearch, and offers a mapping of configuration settings between X-Pack and OpenSearch for various certificate types and security features.
X-Pack Security and OpenSearch Security offer similar features for securing your cluster. This article describes how to migrate from X-Pack Security to OpenSearch Security.
Introduction
Both X-Pack Security and OpenSearch Security provide a range of tools and features for securing your cluster. This makes migration straightforward for most use cases. You should be able to set up a similar setup with OpenSearch Security as with X-Pack Security. First, let’s have a look at how communication is encrypted for both platforms.
TLS Encryption
Similarities
You can use TLS encryption for both the REST API layer (“HTTPS”) and the transport layer. The first encrypts calls to the REST API. The latter is used for encrypting communication in your cluster when the nodes talk to each other.
Both platforms require TLS for the transport layer, while TLS on the REST layer is optional.
Both platforms support the following formats:
X509 certificates / PEM
Java Key Stores (JKS)
PKCS12/PFX
While OpenSearch prefers PEM certificates, the Elasticsearch certutil by default generates PKCS#12 files. But don’t worry, there are several ways to cope with that.
Differences
OpenSearch has two conceptual differences from X-Pack that we need to address:
Node certificates
While X-Pack is happy to let any node join your cluster that has a valid TLS certificate, OpenSearch is a bit more strict here. OpenSearch only allows trusted nodes to join the cluster. Trusted nodes are identified by the Distinguished Name (“DN”) entry of their TLS certificate. Valid DNs need to be listed in opensearch.yml. This should not be a problem, as we only need to figure out the DNs of the certificates already in use and add it to opensearch.yml.
Admin certificate
While X-Pack uses the concept of a root user for changing critical security settings, OpenSearch uses a TLS client certificate that grants access to the security configuration. In OpenSearch, a regular user is never allowed to change the overall security configuration of your cluster.
This means that compared to X-Pack we need to generate one additional TLS certificate for our OpenSearch cluster. The certificate must be signed by the same root CA as your node certificates.
Depending on how you initially generated your certificates, the necessary steps may vary.
DN of Node Certificates
For defining which nodes are allowed to join our OpenSearch cluster, we need to get the DN of our currently used certificates.
JKS / PKCS#12
If you currently use JKS or PKCS#12 files, one of the easiest ways to inspect its contents is to use the Keystore Explorer Tool. You can open your JKS or PKCS#12 file directly and inspect its contents. For example, if we inspect a PKCS#12 file generated by the Elasticsearch certutil tool, we see two entries, “ca” and “instance”:
The ca entry represents the root CA which was used to sign the “instance” certificate. If we click on the instance certificate, we can inspect the DN:
If we want to use this certificate on an OpenSearch cluster, we just need to add the DN in opensearch.yml like:
copy
plugins.security.nodes_dn:
- 'CN=instance'
Note that you can also use wildcards and regular expressions here. If you plan to use individual certificates for each node, this comes in handy. Example:
copy
plugins.security.nodes_dn:
- 'CN=node.com,OU=SSL,O=Test,L=Test,C=DE'
- 'CN=*.example.com,OU=SSL,O=Test,L=Test,C=DE'
- 'CN=elk-devcluster*'
- '/CN=.*regex/'
For JKS files you can also use the Java keytool to inspect the certificate details.
X509 / PEM
To view the contents of a pem file, we recommend using openssl. Note that the file may end in “.pem” or “.crt” both suffixes are commonly used.
copy
openssl x509 -in certificate.crt -text -noout
This will print out the details of the certificate in human-readable form. Again, we are looking for the DN of the certificate which you will find in the “subject” field:
copy
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 1524368626614 (0x162eb7353b6)
Signature Algorithm: sha256WithRSAEncryption
Issuer: DC=com, DC=example, O=Example Com Inc., OU=Example Com Inc. Root CA, CN=Example Com Inc. Root CA
Validity
Not Before: Apr 22 03:43:47 2018 GMT
Not After : Apr 19 03:43:47 2028 GMT
Subject: DC=de, L=test, O=node, OU=node, CN=node-0.example.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
... more ...
So in this example the DN of the certificate is
copy
DC=com, L=test, O=node, OU=node, CN=node-0.example.com
As in the example before, we add this DN to opensearch.yml in the nodes_dn section:
copy
plugins.security.nodes_dn:
- 'DC=com, L=test, O=node, OU=node, CN=node-0.example.com'
Admin Certificates
To gain root access to the cluster and to set up our security configuration, we also need an admin TLS certificate. As noted before, this is a different approach than Elasticsearch implements, so you will likely need to generate a new certificate. The new certificate must be signed with the same CA as your node certificates. However, it is not possible to reuse a node certificate as an admin certificate!
OpenSearch does not have any special requirements for an admin certificate, so we just need to generate a normal TLS client certificate. There are several ways of creating a new certificate. If you are familiar with the Elasticsearch certutil, generating an additional certificate is easy.
copy
./bin/elasticsearch-certutil cert --ca=elastic-stack-ca.p12 --pem --name=admin
the cert command tells certutil that we want a regular TLS certificate
the –ca option tells certutil that we want to use the root CA in the file elastic-stack-ca.p12 for signing
the –pem option is used to create X509 certificates
finally, –name defines the name for this certificate. This will then become the DN
If you run certutil the first time, it will generate the root CA and place it in the default file elastic-stack-ca.p12. You can then use this CA later to generate as many certificates as you need.
After running this command a “certificate-bundle.zip” is created which contains two files, the actual certificate, and the private key. As with the node certificates, we need to find out the DN of the certificate to give it root access to the cluster. With certutil, it will be the name we specified on the command line, prefixed with “CN=”. So in our case, the DN is “CN=admin”. To verify, use openssl to inspect the certificate details as described above.
The last thing we need to do is give the certificate root access by configuring it in opensearch.yml:
copy
searchguard.authcz.admin_dn:
- CN=admin
There are of course other ways of creating certificates:
Using OpenSSL as described in the OpenSearch docs
Using third-party tools like the free Search Guard TLS tool
Configuration Mapping
Understanding and working around the differences between the two platforms was the hard part. Now let’s have a look at the differences in the TLS configuration in elasticsearch.yml and opensearch.yml. Since TLS is standardized, luckily the differences are pretty minimal and boil down to changes in the naming of the configuration keys. We compiled the following tables with the most common configuration settings for your reference:
PEM Certificates: Transport Layer
| X-Pack | OpenSearch |
|---|---|
| xpack.security.transport.ssl.enabled | None. Transport TLS is mandatory |
| xpack.security.transport.ssl.key | plugins.security.ssl.transport.pemkey_filepath |
| xpack.security.transport.ssl.key_passphrase | plugins.security.ssl.transport.pemkey_password |
| xpack.security.transport.ssl.certificate | plugins.security.ssl.transport.pemcert_filepath |
| xpack.security.transport.ssl.certificate_authorities | plugins.security.ssl.transport.pemtrustedcas_filepath |
PEM Certificates: REST Layer
| X-Pack | OpenSearch |
|---|---|
| xpack.security.http.ssl.enabled | plugins.security.ssl.http.enabled |
| xpack.security.http.ssl.key | plugins.security.ssl.http.pemkey_filepath |
| xpack.security.http.ssl.key_passphrase | plugins.security.ssl.http.pemkey_password |
| xpack.security.http.ssl.certificate | plugins.security.ssl.http.pemcert_filepath |
| xpack.security.http.ssl.certificate_authorities | plugins.security.ssl.http.pemtrustedcas_filepath |
JKS and PCKCS12/PXF: Transport Layer
| X-Pack | OpenSearch |
|---|---|
| xpack.security.transport.ssl.keystore.path | plugins.security.ssl.transport.keystore_filepath |
| xpack.security.transport.ssl.keystore.password | plugins.security.ssl.transport.keystore_password |
| xpack.security.transport.ssl.keystore.key_password | plugins.security.ssl.transport.keystore_keypassword |
| xpack.security.transport.ssl.truststore.path | plugins.security.ssl.transport.truststore_filepath |
| xpack.security.transport.ssl.truststore.password | plugins.security.ssl.transport.truststore_password |
JKS and PCKCS12/PXF: REST Layer
| X-Pack | OpenSearch |
|---|---|
| xpack.security.http.ssl.keystore.path | plugins.security.ssl.http.keystore_filepath |
| xpack.security.http.ssl.keystore.password | plugins.security.ssl.http.keystore_password |
| xpack.security.http.ssl.keystore.key_password | plugins.security.ssl.http.keystore_keypassword |
| xpack.security.http.ssl.truststore.path | plugins.security.ssl.http.truststore_filepath |
| xpack.security.http.ssl.truststore.password | plugins.security.ssl.http.truststore_password |
Validation Settings
| X-Pack | OpenSearch |
|---|---|
| xpack.security.transport.ssl.verification_mode | plugins.security.ssl.transport.enforce_hostname_verification |
| xpack.security.transport.ssl.verification_mode | plugins.security.ssl.transport.resolve_hostname |
Note:
X-Pack supports three values for “verification_mode”: full, certificate, and none.
If you do not configure anything regarding certificate validation, OpenSearch uses what’s called “certificate” in X-Pack.
If you set enforce_hostname_verification to true, OpenSearch uses what’s called “full” in X-Pack.
X-Pack does not support hostname resolution, so there is no equivalent in X-Pack for “resolve_hostname”.
OpenSearch does not support “none” for verification mode, since this means all certificates are trusted, even if they are invalid.
TLS Client Authentication
| X-Pack | OpenSearch |
|---|---|
| - | plugins.security.ssl.http.clientauth_mode |
Note:
OpenSearch supports three different settings for HTTP client authentication: NONE, OPTIONAL, and REQUIRE. If you want to mimick the behaviour of X-Pack, set it to OPTIONAL.
If set to NONE, TLS client authentication is disabled. If set to REQUIRE, then any HTTP request needs to carry a TLS client certificate. If it does not, the request is denied.
Ciphers and Protocols
| X-Pack | OpenSearch |
|---|---|
| xpack.security.http.ssl.supported_protocols | plugins.security.ssl.http.enabled_protocols |
| xpack.security.http.ssl.cipher_suites | plugins.security.ssl.http.enabled_ciphers |
| xpack.security.transport.ssl.supported_protocols | plugins.security.ssl.transport.enabled_protocols |
| xpack.security.transport.ssl.cipher_suites | plugins.security.ssl.transport.enabled_ciphers |
What’s Next?
In our next article (coming soon) we will explore how to migrate authentication and authorization.
In the meantime, you can also dig into our general Elasticsearch to OpenSearch migration articles using:
Ready to get started?!
Let's work together to navigate your OpenSearch journey. Send us a message and talk to the team today!
Get in touch