OpenSearch

2022-07-28

From X-Pack to OpenSearch: Part 1 - TLS Setup

This article describes how to migrate TLS Settings from X-Pack Security to OpenSearch Security.

X-Pack Security and OpenSearch Security offer similar features for securing your cluster. This article describes how to migrate from X-Pack Security to OpenSearch Security.

Introduction

Both X-Pack Security and OpenSearch Security provide a range of tools and features for securing your cluster. This makes migration straightforward for most use cases. You should be able to set up a similar setup with OpenSearch Security as with X-Pack Security. First, let’s have a look at how communication is encrypted for both platforms.

TLS Encryption

Similarities

You can use TLS encryption for both the REST API layer (“HTTPS”) and the transport layer. The first encrypts calls to the REST API. The latter is used for encrypting communication in your cluster when the nodes talk to each other.

Both platforms require TLS for the transport layer, while TLS on the REST layer is optional.

Both platforms support the following formats:

X509 certificates / PEM

Java Key Stores (JKS)

PKCS12/PFX

While OpenSearch prefers PEM certificates, the Elasticsearch certutil by default generates PKCS#12 files. But don’t worry, there are several ways to cope with that.

Differences

OpenSearch has two conceptual differences from X-Pack that we need to address:

Node certificates

While X-Pack is happy to let any node join your cluster that has a valid TLS certificate, OpenSearch is a bit more strict here. OpenSearch only allows trusted nodes to join the cluster. Trusted nodes are identified by the Distinguished Name (“DN”) entry of their TLS certificate. Valid DNs need to be listed in opensearch.yml. This should not be a problem, as we only need to figure out the DNs of the certificates already in use and add it to opensearch.yml.

Admin certificate

While X-Pack uses the concept of a root user for changing critical security settings, OpenSearch uses a TLS client certificate that grants access to the security configuration. In OpenSearch, a regular user is never allowed to change the overall security configuration of your cluster.

This means that compared to X-Pack we need to generate one additional TLS certificate for our OpenSearch cluster. The certificate must be signed by the same root CA as your node certificates.

Depending on how you initially generated your certificates, the necessary steps may vary.

DN of Node Certificates

For defining which nodes are allowed to join our OpenSearch cluster, we need to get the DN of our currently used certificates.

JKS / PKCS#12

If you currently use JKS or PKCS#12 files, one of the easiest ways to inspect its contents is to use the Keystore Explorer Tool. You can open your JKS or PKCS#12 file directly and inspect its contents. For example, if we inspect a PKCS#12 file generated by the Elasticsearch certutil tool, we see two entries, “ca” and “instance”:

The ca entry represents the root CA which was used to sign the “instance” certificate. If we click on the instance certificate, we can inspect the DN:

If we want to use this certificate on an OpenSearch cluster, we just need to add the DN in opensearch.yml like:

plugins.security.nodes_dn:
  - 'CN=instance'

Note that you can also use wildcards and regular expressions here. If you plan to use individual certificates for each node, this comes in handy. Example:

plugins.security.nodes_dn:
  - 'CN=node.com,OU=SSL,O=Test,L=Test,C=DE'
  - 'CN=*.example.com,OU=SSL,O=Test,L=Test,C=DE'
  - 'CN=elk-devcluster*'
  - '/CN=.*regex/'

For JKS files you can also use the Java keytool to inspect the certificate details.

X509 / PEM

To view the contents of a pem file, we recommend using openssl. Note that the file may end in “.pem” or “.crt” both suffixes are commonly used.

openssl x509 -in certificate.crt -text -noout

This will print out the details of the certificate in human-readable form. Again, we are looking for the DN of the certificate which you will find in the “subject” field:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1524368626614 (0x162eb7353b6)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: DC=com, DC=example, O=Example Com Inc., OU=Example Com Inc. Root CA, CN=Example Com Inc. Root CA
        Validity
            Not Before: Apr 22 03:43:47 2018 GMT
            Not After : Apr 19 03:43:47 2028 GMT
        Subject: DC=de, L=test, O=node, OU=node, CN=node-0.example.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                ... more ...

So in this example the DN of the certificate is

DC=com, L=test, O=node, OU=node, CN=node-0.example.com

As in the example before, we add this DN to opensearch.yml in the nodes_dn section:

plugins.security.nodes_dn:
  - 'DC=com, L=test, O=node, OU=node, CN=node-0.example.com'

Admin Certificates

To gain root access to the cluster and to set up our security configuration, we also need an admin TLS certificate. As noted before, this is a different approach than Elasticsearch implements, so you will likely need to generate a new certificate. The new certificate must be signed with the same CA as your node certificates. However, it is not possible to reuse a node certificate as an admin certificate!

OpenSearch does not have any special requirements for an admin certificate, so we just need to generate a normal TLS client certificate. There are several ways of creating a new certificate. If you are familiar with the Elasticsearch certutil, generating an additional certificate is easy.

./bin/elasticsearch-certutil cert --ca=elastic-stack-ca.p12 --pem --name=admin

the cert command tells certutil that we want a regular TLS certificate

the –ca option tells certutil that we want to use the root CA in the file elastic-stack-ca.p12 for signing

the –pem option is used to create X509 certificates

finally, –name defines the name for this certificate. This will then become the DN

If you run certutil the first time, it will generate the root CA and place it in the default file elastic-stack-ca.p12. You can then use this CA later to generate as many certificates as you need.

After running this command a “certificate-bundle.zip” is created which contains two files, the actual certificate, and the private key. As with the node certificates, we need to find out the DN of the certificate to give it root access to the cluster. With certutil, it will be the name we specified on the command line, prefixed with “CN=”. So in our case, the DN is “CN=admin”. To verify, use openssl to inspect the certificate details as described above.

The last thing we need to do is give the certificate root access by configuring it in opensearch.yml:

searchguard.authcz.admin_dn:
  - CN=admin

There are of course other ways of creating certificates:

Using OpenSSL as described in the OpenSearch docs

Using third-party tools like the free Search Guard TLS tool

Configuration Mapping

Understanding and working around the differences between the two platforms was the hard part. Now let’s have a look at the differences in the TLS configuration in elasticsearch.yml and opensearch.yml. Since TLS is standardized, luckily the differences are pretty minimal and boil down to changes in the naming of the configuration keys. We compiled the following tables with the most common configuration settings for your reference:

PEM Certificates: Transport Layer

X-Pack	OpenSearch
xpack.security.transport.ssl.enabled	None. Transport TLS is mandatory
xpack.security.transport.ssl.key	plugins.security.ssl.transport.pemkey_filepath
xpack.security.transport.ssl.key_passphrase	plugins.security.ssl.transport.pemkey_password
xpack.security.transport.ssl.certificate	plugins.security.ssl.transport.pemcert_filepath
xpack.security.transport.ssl.certificate_authorities	plugins.security.ssl.transport.pemtrustedcas_filepath

PEM Certificates: REST Layer

X-Pack	OpenSearch
xpack.security.http.ssl.enabled	plugins.security.ssl.http.enabled
xpack.security.http.ssl.key	plugins.security.ssl.http.pemkey_filepath
xpack.security.http.ssl.key_passphrase	plugins.security.ssl.http.pemkey_password
xpack.security.http.ssl.certificate	plugins.security.ssl.http.pemcert_filepath
xpack.security.http.ssl.certificate_authorities	plugins.security.ssl.http.pemtrustedcas_filepath

JKS and PCKCS12/PXF: Transport Layer

X-Pack	OpenSearch
xpack.security.transport.ssl.keystore.path	plugins.security.ssl.transport.keystore_filepath
xpack.security.transport.ssl.keystore.password	plugins.security.ssl.transport.keystore_password
xpack.security.transport.ssl.keystore.key_password	plugins.security.ssl.transport.keystore_keypassword
xpack.security.transport.ssl.truststore.path	plugins.security.ssl.transport.truststore_filepath
xpack.security.transport.ssl.truststore.password	plugins.security.ssl.transport.truststore_password

JKS and PCKCS12/PXF: REST Layer

X-Pack	OpenSearch
xpack.security.http.ssl.keystore.path	plugins.security.ssl.http.keystore_filepath
xpack.security.http.ssl.keystore.password	plugins.security.ssl.http.keystore_password
xpack.security.http.ssl.keystore.key_password	plugins.security.ssl.http.keystore_keypassword
xpack.security.http.ssl.truststore.path	plugins.security.ssl.http.truststore_filepath
xpack.security.http.ssl.truststore.password	plugins.security.ssl.http.truststore_password

Validation Settings

X-Pack	OpenSearch
xpack.security.transport.ssl.verification_mode	plugins.security.ssl.transport.enforce_hostname_verification
xpack.security.transport.ssl.verification_mode	plugins.security.ssl.transport.resolve_hostname

Note:

X-Pack supports three values for “verification_mode”: full, certificate, and none.

If you do not configure anything regarding certificate validation, OpenSearch uses what’s called “certificate” in X-Pack.

If you set enforce_hostname_verification to true, OpenSearch uses what’s called “full” in X-Pack.

X-Pack does not support hostname resolution, so there is no equivalent in X-Pack for “resolve_hostname”.

OpenSearch does not support “none” for verification mode, since this means all certificates are trusted, even if they are invalid.

TLS Client Authentication

X-Pack	OpenSearch
-	plugins.security.ssl.http.clientauth_mode

Note:

OpenSearch supports three different settings for HTTP client authentication: NONE, OPTIONAL, and REQUIRE. If you want to mimick the behaviour of X-Pack, set it to OPTIONAL.

If set to NONE, TLS client authentication is disabled. If set to REQUIRE, then any HTTP request needs to carry a TLS client certificate. If it does not, the request is denied.

Ciphers and Protocols

X-Pack	OpenSearch
xpack.security.http.ssl.supported_protocols	plugins.security.ssl.http.enabled_protocols
xpack.security.http.ssl.cipher_suites	plugins.security.ssl.http.enabled_ciphers
xpack.security.transport.ssl.supported_protocols	plugins.security.ssl.transport.enabled_protocols
xpack.security.transport.ssl.cipher_suites	plugins.security.ssl.transport.enabled_ciphers

What’s Next?

In our next article (coming soon) we will explore how to migrate authentication and authorization.

In the meantime, you can also dig into our general Elasticsearch to OpenSearch migration articles using:

Snapshot/Restore

Reindex API

Rolling Restart