Adding a customer managed encryption key to a CDP environment running on AWS

By default, Data Lake and FreeIPA's Amazon Elastic Block Store (EBS) volumes and Relational Database Service (RDS) are encrypted using a default key from Amazon’s KMS, but you can optionally configure encryption using Customer Managed Keys (CMK). Data Hubs inherit environment's encryption key by default but you have an option to specify a different CMK during Data Hub creation.

Amazon offers the option to encrypt EBS volumes and RDS instances using a default key from Amazon's Key Management System (KMS) or using an external customer-managed KMS. By default, Data Lake and FreeIPA are encrypted using the default key from Amazon’s KMS present in the region where the environment is running, but you can provide a customer-managed KMS key instead of the default key.

By default, Data Hubs use the same default key from Amazon’s KMS or CMK as the parent environment but you have an option to pass a different CMK during Data Hub creation.

Encryption is configured for block devices and root devices. When encryption is configured for a given cluster, it is automatically applied to all the disk devices of any new VM instances added as a result of cluster scaling or repair.

This documentation covers the following topics:
  • AWS disk encryption options
  • AWS prerequisites for using a CMK
  • Registering a new AWS environment and specifying a CMK
  • Adding a CMK to an existing AWS environment

Register an AWS environment with a CMK

You can specify an existing customer managed key (CMK) during AWS environment registration and the encryption key will be used to encrypt the EBS volumes and RDS instances running in the environment.

Steps

You can register your environment as described in Register an AWS environment from CDP UI, just make sure that on the Data Access and Audit page you enable the following:
  1. Under Customer-Managed Keys, click Enable Customer-Managed Keys.

  2. In the same section, select the CMK:

The following screenshot shows the UI options:

You can use your usual CDP CLI command to create an environment with a CMK, just add the --encryption-key-arn parameter and provide the encryption key created earlier.

Use the following CDP CLI command to create an environment with the encryption key created earlier. Replace the placeholders with actual values. For example <ENVIRONMENT-NAME> should be replaced with an actual name. The parameter important for this feature are highlighted:
cdp environments create-aws-environment \
--environment-name <ENVIRONMENT-NAME> \
--credential-name <EXISTING_CREDENTIAL> \
--region "<REGION>" \
--security-access cidr=<CIDR> \
--authentication publicKeyId="<SSH_KEY>" \
--log-storage storageLocationBase=<BUCKET_URL>,instanceProfile=<IDBROKER_IP> \
--vpc-id <VPC_ID> \
--subnet-ids <SUBNETS \
--encryption-key-arn <ENCRYPTION_KEY_ARN>

The ARN of the encryption key created earlier should be passed in the parameter --encryption-key-arn

If the customer-managed encryption key ARN is not passed, then the AWS region-specific default encryption key is used for encrypting EBS volumes and RDS instances.

You can obtain more complete commands using the instructions in Obtain CLI commands for registering an environment.

Set a CMK for an existing AWS environment

You can set a CMK for an existing environment. The CMK will be only used for encrypting disks of Data Hubs created after the CMK was added.

Steps

You can add an encryption key for an existing environment that does not have an encryption key set by navigating to the Summary page of the environment.
  1. In the Management Console, navigate to Environments and click on the environment for which you would like to set a CMK.

  2. Click on the Summary tab.

  3. Scroll down to the Customer Managed Encryption Key section.

  4. If no CMK has been set, you will see a message stating that there is no customer-managed key enabled.

  5. Click on the edit button in the top right corner of the tab.

  6. Click on the toggle button next to Enable Customer-Managed Keys to enable adding a CMK.

  7. Provide the encryption key ARN. You can copy and paste it from AWS KMS:

  8. Click Save.

You can add an encryption key for an existing environment that does not yet have encryption enabled.

cdp environments update-aws-disk-encryption-parameters \
  --environment-name <ENVIRONMENT_NAME> 
  --encryption-key-arn <CMK_ARN>