From X-Pack to OpenSearch: Part 3 – Migrating Custom Roles

OpenSearch

2022-09-22

From X-Pack to OpenSearch: Part 3 – Migrating Custom Roles

This article describes how to migrate custom roles from X-Pack Security to OpenSearch Security.

This article describes how to migrate custom roles from X-Pack Security to OpenSearch Security.

X-Pack Roles

In X-Pack there are privileges which are a named pre-defined set of actions for operating on indices or the Elasticsearch cluster. For example write is an index level privilege granting write operations on documents stored in indices. The cluster level privilege create_snapshot would allow creating snapshots and to list existing repositories. Privileges are the lowest level of granularity in X-Pack Security - you can not change the pre-defined privileges or operate directly on the action level.
One or more privileges are then combined with a secured resource (typically an index or index pattern) into a permission. For example, we define a permission which grants the read and write privileges to all indices matching the my-index* pattern.
A role, which can be assigned to users via role mapping, is a named set of permissions and look like this:
{
  "cluster":[
    "all"
  ],
  "indices":[
    {
      "names":[
        "index*",
        "some"
      ],
      "privileges":[
        "all"
      ],
      "allow_restricted_indices":false
    },
    {
      "names":[
        "index1*",
        "some1"
      ],
      "privileges":[
        "all"
      ],
      "allow_restricted_indices":false
    },
    {
      "names":[
        "index2*",
        "some1"
      ],
      "privileges":[
        "read"
      ],
      "allow_restricted_indices":false
    }
  ]
}
Roles and role mappings can be retrieved through the X-Pack Security APIs or by querying the .security-7 index described in our last blog post. That is what we do in this blog post.

OpenSearch Security Roles

In OpenSearch Security, roles, mappings, permissions, and actions exist. Contrary to X-Pack Security you can define permissions also on an action level. Pre-defined sets of actions are not called privileges but action groups. This is outlined here:
X-Pack Security OpenSearch Security Consists of Remarks
action* action - Only in OpenSearch Security
privilege action group actions*  
permission permission privileges or action groups  
role role permissions  
role mapping roles mapping roles and users  
*Not directly configurable in X-Pack Security
A role in OpenSearch Security looks like this:
{
  "cluster_permissions":[
    "cluster_all"
  ],
  "index_permissions":[
    {
      "index_patterns":[
        "index*",
        "some"
      ],
      "allowed_actions":[
        "indices_all"
      ]
    },
    {
      "index_patterns":[
        "index1*",
        "some1"
      ],
      "allowed_actions":[
        "indices_all"
      ]
    },
    {
      "index_patterns":[
        "index2*",
        "some1"
      ],
      "allowed_actions":[
        "read"
      ]
    }
  ]
}

Migrate Roles

To migrate our custom roles we need to retrieve the roles from Elasticsearch, change the syntax and structure, map the privilege names to action groups and import the role into OpenSearch. Within the scope of this article, we outline some basic concepts to accomplish this. To deal with JSON output on the command line, we recommend that you install a tool called jq and use the bash shell for the commands stated below.
Let’s issue a curl request to retrieve all our Elasticsearch custom roles:
curl -Ss -u "elastic:<password>" localhost:9200/.security-7/_search?pretty\&size=1000\&q=type:role 
Now we leverage jq to make some changes to the roles to make them look more like what we need to import them into OpenSearch:
curl -Ss -u "elastic:<password>" localhost:9200/.security-*/_search?pretty\&size=1000\&q=type:role | jq '.hits.hits[] | { _rolename: ._id | split("-")[1], cluster_permissions: ._source.cluster, index_permissions: ._source.indices } | del(.index_permissions[].allow_restricted_indices)
This yields a result like:
{
  "_rolename": "my_custom_role",
  "cluster_permissions": [
    "all"
  ],
  "index_permissions": [
    {
      "names": [
        "index*",
        "some"
      ],
      "privileges": [
        "all"
      ]
    },
    {
      "names": [
        "index1*",
        "some1"
      ],
      "privileges": [
        "all"
      ]
    },
    {
      "names": [
        "index2*",
        "some1"
      ],
      "privileges": [
        "read"
      ]
    }
  ]
}

Not bad but we also need to rename some JSON fields and map the privilege names to pre-defined default action groups.
Let’s add some bash scripting to tweak the output:
# Save the results from above in a file called roles_mig.tmp
curl -Ss -u "elastic:<password>" localhost:9200/.security-*/_search?pretty\&size=1000\&q=type:role | jq '[.hits.hits[] | { _rolename: ._id | split("-")[1], cluster_permissions: ._source.cluster, index_permissions: ._source.indices } | del(.index_permissions[].allow_restricted_indices) | del(.index_permissions[].allow_restricted_indices)]' > roles_mig_tmp.json

# Loop roles_mig.tmp (every row is a role)
for row in $(cat "roles_mig_tmp.json" | jq -r '.[] | @base64'); do

   # get the count of how many index permissions we have in this role
   index_permissions_count=$(echo ${row} | base64 --decode | jq  '.index_permissions | length')
   let index_permissions_count=index_permissions_count-1

   JQ_NAME_FIX=""
   for i in $(eval echo "{0..$index_permissions_count}")
   do
     # Rename .names to .index_patterns and .privileges to .allowed_actions
     JQ_NAME_FIX=".index_permissions[$i].index_patterns=.index_permissions[$i].names | $JQ_NAME_FIX "
     JQ_NAME_FIX=".index_permissions[$i].allowed_actions=.index_permissions[$i].privileges | $JQ_NAME_FIX "
   done

   # TODO
   # Map privilege names to action groups and/or create custom action groups

	# Extract the role name
   ROLE_NAME=$(echo ${row} | base64 --decode | jq  '._rolename' | tr -d '"')

   # Remove all fields we do not longer need like ._rolename or .privileges
   JSON="$(echo ${row} | base64 --decode | jq  "$JQ_NAME_FIX del(.index_permissions[].names) | del(.index_permissions[].privileges) | del(._rolename)")"

   echo "curl -k -u opensearch_admin:<password> -X PUT https://localhost:9200/_plugins/_security/api/roles/$ROLE_NAME -H 'content-type: application/json' -d '$(echo ${JSON} | jq .)'"
done

The output are almost ready-to-use curl commands to create roles by using the Create role API:
curl -k -u opensearch_admin:<password> -X PUT https://localhost:9200/_plugins/_security/api/roles/my_other_role2 -H 'content-type: application/json' -d '{
  "cluster_permissions": [
    "all"
  ],
  "index_permissions": [
    {
      "allowed_actions": [
        "all"
      ],
      "index_patterns": [
        "index*",
        "some"
      ]
    },
    {
      "allowed_actions": [
        "all"
      ],
      "index_patterns": [
        "index1*",
        "some1"
      ]
    },
    {
      "allowed_actions": [
        "read"
      ],
      "index_patterns": [
        "index2*",
        "some1"
      ]
    }
  ]
}'
Note: The script does NOT replace the privilege names automatically! This is something you need to do manually like described in the next chapter.

Mapping Privileges to Action Groups

In most cases, it should be too hard to map X-Pack Security privileges to OpenSearch Security default action groups. Keep in mind that, even if the name of the privilege and action group sounds similar, the granted actions might not be exactly the same. We recommend reviewing every privilege in the original Elasticsearch role and double-checking that the corresponding action group will at least not permit more permissions than the original privilege.
If there are no default action groups that match your specific needs, you can of course replace a privilege with more than one action group or define a new action group. See also OpenSearch Security Part 1: Concepts.

Cluster Privileges

X-Pack Security privilege OpenSearch Security default action group
all cluster_all
create_snapshot manage_snapshots
manage_index_templates cluster_manage_index_templates
manage_ingest_pipelines delete
index write
manage manage
manage_pipeline cluster_manage_pipelines
monitor cluster_monitor

Indices Privileges

X-Pack Security privilege OpenSearch Security default action group
all indices_all
create index
create_index create_index
delete delete
index write
manage manage
monitor indices_monitor
read read
write write + delete

Next Steps

In our next article, we will cover the migration of more complex roles containing field- and document-level security as well as user impersonation, tenants and role mappings.
Get in touch with us
Send us a message and talk to the team today.
Need help with your setup?
OpenSearch or Elastic Stack – We’ve got you covered.