OpenSearch

2021-08-18

OpenSearch Security Part 2: Basic Setup

In this article we will walk you through a basic OpenSearch security setup using the demo configuration and the internal user database.

In the previous article of this series, we looked at the basic concepts of OpenSearch security. In this part, we will walk you through the steps of setting up basic security controls based on users, roles, and permissions.

Prerequisites

As a prerequisite, download and install the standard (non-minimal) distribution of OpenSearch. This distribution comes with a lot of plugins already pre-installed, including the security plugin.

As discussed in the previous article, setting up security controls on OpenSearch involves:

Setting up TLS encryption between OpenSearch nodes and optionally on REST

Configuring an Admin TLS certificate for full access to the cluster

Defining users and roles for fine-grained access control

OpenSearch ships a demo installer that takes care of all of that. After running it, you have encryption and a set of demo users and roles already installed.

Word of warning: The demo installer uses self-signed certificates that ship with OpenSearch. Do not use them in production!

Installing the Demo Configuration

To execute the demo installer, first, go to the installation directory of OpenSearch. Then change to:

cd plugins/opensearch-security/tools/

In this directory you will find tools for administering the security setup, including the demo installer. Depending on your installation method, you may need to chmod the installer first. We answer the questions like:

./install_demo_configuration.sh 

OpenSearch Security Demo Installer
 ** Warning: Do not use on production or public reachable systems **
Install demo certificates? [y/N] y
Initialize Security Modules? [y/N] y
Cluster mode requires maybe additional setup of:
  - Virtual memory (vm.max_map_count)

Enable cluster mode? [y/N] N

We tell the installer that we want to install the built-in demo TLS certificates (again, do not use them in production!) and also initialize the security module with the demo users and roles. Since we just run OpenSearch as a PoC on our local system, we do not need to enable cluster mode.

After the script has been executed successfully, it prints out some information about the OpenSearch installation:

OpenSearch install type: .tar.gz on 
OpenSearch config dir: /Users/myuser/Development/opensearch-1.0.0/config
OpenSearch config file: /Users/myuser/Development/opensearch-1.0.0/config/opensearch.yml
OpenSearch bin dir: /Users/myuser/Development/opensearch-1.0.0/bin
OpenSearch plugins dir: /Users/myuser/Development/opensearch-1.0.0/plugins
OpenSearch lib dir: /Users/myuser/Development/opensearch-1.0.0/lib
Detected OpenSearch Version: x-content-1.0.0
Detected OpenSearch Security Version: 1.0.0.0

Installed Artefacts

TLS Certificates

Let’s examine what the script actually changed and installed. First, we have a look at the config directory. The following files have been added:

root-ca.pem
esnode.pem
esnode-key.pem
kirk.pem
kirk-key.pem

root-ca.pem: This is the certificate of the root CA that signed all other TLS certificates

esnode.pem: This is the certificate that this node uses when communicating with other nodes on the transport layer (inter-node traffic) esnode-key.pem: The private key for the esnode.pem node certificate

kirk.pem: This is the admin TLS certificate used when making changes to the security configuration. This certificate gives you full access to the cluster

kirk-key.pem: The private key for the admin TLS certificate

opensearch.yml

The installer also made changes to the opensearch.yml configuration file and added sensible defaults.

In the first section, the TLS certificates for inter-node encryption are configured. Note that the path to the certificates and keys has to be specified relative to the config directory. TLS hostname verification is disabled because we use self-signed certificates.

plugins.security.ssl.transport.pemcert_filepath: esnode.pem
plugins.security.ssl.transport.pemkey_filepath: esnode-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: root-ca.pem
plugins.security.ssl.transport.enforce_hostname_verification: false

TLS for the REST layer (“HTTPS”) is enabled as well. Note that the installer uses the same self-signed certificates as above. This is fine for a PoC. In production, you would rather use certificates from a well-known CA, like Let’s Encrypt, Thawte, DigiCert, etc.

plugins.security.ssl.http.enabled: true
plugins.security.ssl.http.pemcert_filepath: esnode.pem
plugins.security.ssl.http.pemkey_filepath: esnode-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: root-ca.pem

For security reasons, we need to tell OpenSearch to accept the demo certificates. The default for this setting is false to make sure you do not run these certificates in production.

plugins.security.allow_unsafe_democertificates: true

We also tell OpenSearch to initialize the security configuration automatically on startup. When this is enabled, OpenSearch first checks if the security module is already initialized. If not, it reads the demo configuration files (see below) and populates the security configuration index with their content:

plugins.security.allow_default_init_securityindex: true

Finally, we tell OpenSearch that the kirk TLS certificate should be granted full access rights for the cluster. For this, we specify the Common Name (CN) for the certificate. It is also possible to define more than one admin certificate here:

plugins.security.authcz.admin_dn:
  - CN=kirk,OU=client,O=client,L=test, C=de

The remainder of the configuration is not relevant for our demo, so we skip it for the moment.

Demo Users and Roles

When our cluster starts up for the first time, the security configuration index does not exist yet. Since we enabled the auto-initialization feature in opensearch.yml, the security module will read all configuration files in the securityconfig directory to create it. This only happens when there is no security index yet. An already existing configuration is never overwritten.

The installer sets up a configuration that uses the internal user database and HTTP Basic for authentication and authorization.

The most important files are:

internal_users.yml: Defines users and their hashed passwords

roles.yml: Defines roles and the associated access permissions

roles_mapping.yml: Defines the mapping from users to roles

We will look at the content of these files in our next article where we add our own users and roles. For now, just remember that the security configuration is stored in the files in the securityconfig directory, and that they are used to initialize the configuration index.

Trying it Out

We have now everything in place to try out our security configuration. Let’s start OpenSearch and see what happens. The first thing you will notice are additional log statements during startup for the TLS setup:

OpenSearch Config path is /Users/myuser/Development/opensearch-1.0.0/config
JVM supports TLSv1.3
Config directory is /Users/myuser/Development/opensearch-1.0.0/config/, from there the key- and truststore files are resolved relatively
TLS Transport Client Provider : JDK
TLS Transport Server Provider : JDK
TLS HTTP Provider             : JDK
Enabled TLS protocols for transport layer : [TLSv1.3, TLSv1.2, TLSv1.1]
Enabled TLS protocols for HTTP layer      : [TLSv1.3, TLSv1.2, TLSv1.1]
...

And we can also see that OpenSearch initialized the security index with the demo configuration files:

Will update 'internalusers' with /Users/myuser/Development/opensearch-1.0.0/plugins/opensearch-security/securityconfig/internal_users.yml 
[.opendistro_security/MkCItfRNQW-KeJHsBesqRA] update_mapping [_doc]
Doc with id 'internalusers' and version 2 is updated in .opendistro_security index.
...

Now, let’s try to access OpenSearch with a Browser via:

http://localhost:9200/

Depending on the browser you are using, you will see an error message and an empty page like:

localhost didn't send any data.

In the OpenSearch logfile we also see an exception like:

Exception during establishing a SSL connection: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record

Why? We have enabled TLS for HTTP connections in opensearch.yml, which means all connections on the REST layer have to be done with HTTPS. However, we tried to access OpenSearch with HTTP. Let’s try again with HTTPS:

https://localhost:9200/

Now we get a different error message, complaining about our root CA. Again, the message varies depending on your Browser, but it should look something like this:

NET::ERR_CERT_AUTHORITY_INVALID

Why does it occur? The TLS certificates that we use are self-signed. This means that the root CA used to sign all other certificates were created by OpenSearch. The Browser now complains that this self-signed root CA is not on the list of trusted CAs. If we would have used a certificate from a well-known CA like Let’s Encrypt instead, we would not see this warning.

Since we know that we use a self-signed certificate, we can add an exception for our root CA.

The Browser will now display an HTTP Basic Authentication dialogue and asks for a username and password. Since we installed the demo configuration, we use admin/admin and can then access the cluster. This shows that the security module is active and that OpenSearch can now be accessed with an authenticated user only.

Next Steps

In the next article, we will describe how to define our own users and roles and how to use the securityadmin tool to change the security configuration.