Cross-account access IAM role

To allow CDP to create resources in your AWS account, you create a cross-account access IAM role in your AWS account and grant CDP access to the role as a trusted principal by specifying a specific AWS account and an external ID.

The policy for the cross-account access IAM role must have the permissions enumerated in the documentation linked below. In addition, the IAM role must reference the specific AWS account ID and external ID provided in the Management Console.

IAM policy definitions

When creating an IAM policy for the provisioning credential, use the following IAM policy definitions.

You have two options:

Option Description Policies
Default policy This policy is appropriate for registering a CDP environment and provisioning all CDP services. This is the same IAM policy as the Default policy provided in the Management Console web interface.
Reduced access policies Instead of the extensive default policy, you can use minimal reduced access policies. Note that the first policy is required for provisioning a CDP environment. The remaining policies are optional and are only needed if you would like to use the specific CDP services.

Create a cross-account IAM role

In order to use role-based authentication, you must create an IAM role on AWS.

Prior to creating a cross-account IAM role on AWS, log in to the CDP web interface and obtain the parameters that need to provide for the IAM role:
  1. In the Management Console, navigate to Environments > Shared Resources > Credentials > Create Credential:
  2. Note or copy the Cross-account Access Policy, Account ID, and External ID listed in the UI:

    You will need them to complete the following steps.

  1. Log in to the AWS Management Console.
  2. Navigate to the IAM console > Roles and click Create Role:
  3. In the “Create Role” wizard, select Another AWS account role type. Next, provide the following:
    • In the Account ID field, copy and paste your Account ID provided in CDP.
    • Under Options, check Require external ID and under External ID, copy and paste the External ID from CDP.
  4. When done, click Next: Permissions to navigate to the next page in the wizard.
  5. Click Create policy and the create policy wizard will open in a new browser tab:
  6. Select the JSON view, and copy and paste the policy definition. You can either copy it from the Management Console web interface or from documentation:
  7. When done, navigate to Review policy.
  8. On the Review policy page, in the Name field, enter a name for your policy:
  9. When done, click Create Policy.
  10. Return to the previous browser tab where you started creating a new role (since the create policy wizard was opened in a new browser tab).
  11. Click Refresh.
  12. Find the policy that you just created and select it by checking the box:
  13. When done, click Next: Review.
  14. In the Roles name field, enter role name.
  15. When done, click Create role to finish the role creation process.
  16. Obtain the IAM Role ARN. You will need it to create a role-based credential:

Once you are done creating the IAM role on AWS, create a role-based credential in CDP.

Reduced access policy definition

The following reduced policy definition can optionally be used to register a CDP environment and create Data Hubs and Operational Databases in it.

The IAM role used for the provisioning credential can use the reduced access policy instead of the default policy.

You should:

  1. Copy this policy.

  2. Update it as described below.

  3. Once done, you can create the cross-account role using this policy.

Customize the scope of iam:PassRole

CDP utilizes the iam:PassRole as part of the cloud identity federation setup process via IDBroker on AWS. The default policy uses “*” as the resource scope for this action:
{
     "Effect": "Allow
      "Action": [
        "iam:PassRole"
      ],
      "Resource": "*"
}

However the scope can be reduced to just a few roles. To do this, you need to edit the following block and replace the “*” with the ARNs of the following roles used for the environment:

  • IDBROKER_ROLE
  • LOG_ROLE
  • RANGER_AUDIT_ROLE
  • DATALAKE_ADMIN_ROLE

You choose the name for these roles when you set up CDP as part of the Minimal Setup for Cloud Storage.

After updating according to the minimal setup for cloud storage, the iam:PassRole policy excerpt should like similar to the following:

{
      "Effect": "Allow
      "Action": [
        "iam:PassRole"
      ],
      "Resource": [
               "arn:aws:iam::$ACCOUNTID:role/IDBROKER_ROLE",
               "arn:aws:iam::$ACCOUNTID:role/LOG_ROLE",
               “arn:aws:iam::$ACCOUNTID:role/RANGER_AUDIT_ROLE",
               "arn:aws:iam::$ACCOUNTID:role/DATALAKE_ADMIN_ROLE"
           ]
}

If you have already created these roles and you don’t know the name of these roles, you can obtain them using the following steps:

IDBROKER_ROLE

  1. Log in to CDP web interface and navigate to the Management Console service > Data Lakes.
  2. Scroll down to where you see Event History and select the Hardware tab.
  3. Locate the EC2 instance for the idbroker node (if you are running Medium Duty Data Lake, you will see two IDBroker nodes, you can use either):
  4. Click on the instance id. You will be redirected to the EC2 console.
  5. In the Instance Details section for this node, you can see the name of the IAM role being used by this instance. Click on this role to get to the details.
  6. Copy the ARN of the instance profile. It follows the following naming convention: arn:aws:iam::<12-digit-AWS-account-id>:instance-profile/<name-of-idbroker-role>

LOG_ROLE

  1. Navigate to Environments .
  2. Click on the existing environment
  3. Click on the Summary tab.
  4. Scroll to Logs Storage and Audits and you will see the Instance Profile entry.
  5. Copy the ARN of the Instance Profile:

RANGER_AUDIT_ROLE and DATALAKE_ADMIN_ROLE

  1. Navigate to Environments.
  2. Click on the existing environment.
  3. From the Actions menu, select Manage Access.
  4. Click on the IDBroker Mappings tab.
  5. Under Current Mappings you will see the mappings for the Data Access Role and Ranger Audit Role.
  6. Copy the ARNs of the Data Access Role and Ranger Audit Role:

Further permissions reduction (Optional)

You can reduce these policies even further by pre-creating some resources:

  • If you set up your own VPC and subnets, then the following actions can be removed from the policy:
    ec2:CreateVpc
    ec2:CreateNatGateway
    ec2:CreateRouteTable
    ec2:CreateSubnet
    ec2:CreateVpcEndpoint
    ec2:CreateInternetGateway
    ec2:DeleteSubnet
    ec2:DeleteInternetGateway
    ec2:AttachInternetGateway
    ec2:DetachInternetGateway
    ec2:DescribePrefixLists
    ec2:AllocateAddress
    ec2:AssociateRouteTable
    ec2:CreateRoute
    ec2:DeleteRouteTable
    ec2:DeleteVpcEndpoints
    ec2:DisassociateRouteTable
    ec2:ReleaseAddress
    ec2:DeleteRoute
    ec2:DeleteNatGateway
    ec2:DeleteVpc
  • If you use your own security groups, then the following actions can be removed from the policy:
    ec2:CreateSecurityGroup
    ec2:AuthorizeSecurityGroupIngress
    ec2:RevokeSecurityGroupEgress
    ec2:AuthorizeSecurityGroupEgress
  • If you use private IPs or set up CCM to communicate with the CDP Control Plane, then the following actions can be removed from the policy:
    ec2:CreateSecurityGroup
    ec2:AuthorizeSecurityGroupIngress
    ec2:RevokeSecurityGroupEgress
    ec2:AuthorizeSecurityGroupEgress
  • If you use private IPs or set up CCM to communicate with the CDP Control Plane, then the following actions can be removed from the policy:
    ec2:AllocateAddress
    ec2:ReleaseAddress
  • If you create your own DynamoDB table for S3Guard, then the following actions can be removed from the policy:
    dynamodb:DeleteTable