Bring Your Own IAM Roles (AWS)¶

By default, qumulo_filesystem_aws creates two IAM roles per deployment (one for the cluster nodes, one for the provisioner VM) and manages their policies for the life of the cluster. If your organization does not allow Terraform credentials to create IAM roles, a security team can create the roles ahead of time and the cluster resource references them by ARN:

resource "qumulo_filesystem_aws" "example" {
  # ... other required fields ...
  cluster_name             = "prodfs"
  deployment_name          = "prodfs"
  cluster_iam_role_arn     = "arn:aws:iam::123456789012:role/prodfs-qumulo-nodes"
  provisioner_iam_role_arn = "arn:aws:iam::123456789012:role/prodfs-qumulo-provisioner"

  timeouts {
    create = "90m"
    delete = "30m"
  }
}

Naming attributes come as a pair

deployment_name must be set together with cluster_name. Combining deployment_name with name is a plan-time error.

The two attributes are independent: bring one role and let the provider manage the other if you like. Changing either forces cluster replacement.

What changes when you bring a role¶

For a role you supply, the provider performs zero IAM writes, ever. You own the role, its policies, and its instance profile.

Operation	Provider-managed role	Bring-your-own role
Create	Creates role, policies, instance profile	Looks up your instance profile (read-only)
Storage scale-up	Rewrites the role's S3 policy for new buckets	Skipped; your wildcard policy already covers them
Tag changes	Tags the role and profile	Skipped
`terraform destroy`	Deletes role, policies, profile	Skipped; your role survives

In return, the deploying credentials shed every IAM write permission the managed path needs (iam:CreateRole, iam:PutRolePolicy, iam:CreateInstanceProfile, iam:TagRole, iam:DeleteRole). They keep two narrow grants, listed in What the deploying credentials need.

Step 0: pick the deployment name¶

Choose deployment_name before creating the roles

The S3 statements in your role policies embed deployment_name, and it cannot change later without replacing the cluster. Pick it first, hand it to whoever writes the policies, and set it explicitly in the resource block so the two visibly agree. Embed it lowercased — S3 bucket names are always lowercase, and the patterns below are case-sensitive globs.

S3 bucket names carry an optional random 11-character prefix and an optional random 11-character suffix around deployment_name, depending on how the cluster was created or imported: natively created buckets get a prefix only, legacy CNQ-imported buckets get both a prefix and a suffix, and some imports carry a suffix only. Your policy cannot know the random values in advance, so it anchors on deployment_name with four patterns that together cover every form (? matches exactly one character, * matches any number):

arn:aws:s3:::<deployment_name>-qps-*
arn:aws:s3:::<deployment_name>-qps-*/*
arn:aws:s3:::<deployment_name>-???????????-qps-*
arn:aws:s3:::<deployment_name>-???????????-qps-*/*
arn:aws:s3:::???????????-<deployment_name>-qps-*
arn:aws:s3:::???????????-<deployment_name>-qps-*/*
arn:aws:s3:::???????????-<deployment_name>-???????????-qps-*
arn:aws:s3:::???????????-<deployment_name>-???????????-qps-*/*

This pattern set also covers buckets created by future storage scale-ups, which the provider cannot add to your policies (it does not write to your roles). Because the patterns are wildcards rather than an exact bucket list, the statement's s3:ResourceAccount condition restricts them to buckets owned by your own AWS account, so they can never reach a same-named bucket squatted in someone else's account.

Create the roles¶

You will create four things: two roles (each with the policies below) and two instance profiles, one per role. Every snippet is copy-paste complete; replace prodfs with your deployment_name and 123456789012 with your account ID.

These documents track the provider

The same policy documents live in the provider repo under examples/aws-byo-iam/policies/, where a CI test keeps them in sync with what the provider-managed path grants. If a provider release adds a permission, that test fails until the example and this guide are updated.

Both roles use this trust policy, and each must live in exactly one instance profile (EC2 attaches roles to instances only through a profile):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "ec2.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}

Cluster node role¶

The nodes run as this role. Attach the AmazonSSMManagedInstanceCore managed policy plus this policy document:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "NodeOps",
      "Effect": "Allow",
      "Action": [
        "cloudwatch:DeleteAlarms",
        "cloudwatch:PutMetricAlarm",
        "ec2:AssignIpv6Addresses",
        "ec2:AssignPrivateIpAddresses",
        "ec2:DescribeInstances",
        "ec2:DescribeNetworkInterfaces",
        "ec2:UnassignIpv6Addresses",
        "ec2:UnassignPrivateIpAddresses",
        "kms:Decrypt",
        "logs:CreateLogStream",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Sid": "StorageKeys",
      "Effect": "Allow",
      "Action": [
        "kms:CreateGrant",
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:GenerateDataKey",
        "kms:GenerateDataKeyWithoutPlaintext",
        "kms:ReEncryptFrom",
        "kms:ReEncryptTo"
      ],
      "Resource": ["<your kms_key_id key ARN>"]
    },
    {
      "Sid": "PersistentStorage",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::prodfs-qps-*",
        "arn:aws:s3:::prodfs-qps-*/*",
        "arn:aws:s3:::prodfs-???????????-qps-*",
        "arn:aws:s3:::prodfs-???????????-qps-*/*",
        "arn:aws:s3:::???????????-prodfs-qps-*",
        "arn:aws:s3:::???????????-prodfs-qps-*/*",
        "arn:aws:s3:::???????????-prodfs-???????????-qps-*",
        "arn:aws:s3:::???????????-prodfs-???????????-qps-*/*"
      ],
      "Condition": {
        "StringEquals": { "s3:ResourceAccount": "123456789012" }
      }
    }
  ]
}

Statement by statement:

Statement	What it is for	Adjust?
`NodeOps`	Node metrics, floating-IP moves, boot logs. These actions do not support resource-level scoping (the managed path also grants them on `*`).	Keep as is
`StorageKeys`	Encryption with a customer-managed KMS key.	Delete it unless you set `kms_key_id`. If you do, put that key's ARN in (one key encrypts both EBS and S3). With default AWS-managed keys, the key policies of `aws/ebs` and `aws/s3` already authorize account principals.
`PersistentStorage`	The cluster's S3 buckets.	Replace `prodfs` with your (lowercased) `deployment_name` and `123456789012` with your account ID; keep the wildcards

TerraformAWS CLI

resource "aws_iam_role" "cluster" {
  name               = "prodfs-qumulo-nodes"
  assume_role_policy = file("trust.json")
}

resource "aws_iam_role_policy_attachment" "cluster_ssm" {
  role       = aws_iam_role.cluster.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_role_policy" "cluster" {
  role   = aws_iam_role.cluster.name
  policy = file("cluster-role-policy.json")
}

# Exactly one instance profile per role.
resource "aws_iam_instance_profile" "cluster" {
  name = aws_iam_role.cluster.name
  role = aws_iam_role.cluster.name
}

aws iam create-role \
  --role-name prodfs-qumulo-nodes \
  --assume-role-policy-document file://trust.json
aws iam attach-role-policy \
  --role-name prodfs-qumulo-nodes \
  --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
aws iam put-role-policy \
  --role-name prodfs-qumulo-nodes \
  --policy-name cluster-policy \
  --policy-document file://cluster-role-policy.json

# Exactly one instance profile per role.
aws iam create-instance-profile \
  --instance-profile-name prodfs-qumulo-nodes
aws iam add-role-to-instance-profile \
  --instance-profile-name prodfs-qumulo-nodes \
  --role-name prodfs-qumulo-nodes

Provisioner role¶

A short-lived orchestration VM runs as this role during create, scaling, node replacement, and configuration updates. Attach two managed policies, AmazonSSMManagedInstanceCore and SecretsManagerReadWrite (the provisioner stores the cluster admin credentials in Secrets Manager), plus this policy document:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Orchestration",
      "Effect": "Allow",
      "Action": [
        "cloudformation:DescribeStackResource",
        "cloudformation:DescribeStackResources",
        "cloudformation:DescribeStacks",
        "cloudformation:SetStackPolicy",
        "cloudformation:UpdateTerminationProtection",
        "ec2:CreateTags",
        "ec2:DeleteTags",
        "ec2:DescribeTags",
        "ec2:DescribeInstances",
        "ec2:DescribeVolumes",
        "ec2:ModifyInstanceAttribute",
        "ec2:ModifyNetworkInterfaceAttribute",
        "ec2:ModifyVolume",
        "kms:Decrypt",
        "ssm:GetParameter",
        "ssm:ListCommandInvocations",
        "ssm:ListInstanceAssociations",
        "ssm:PutParameter",
        "ssm:UpdateInstanceInformation",
        "ssm:SendCommand"
      ],
      "Resource": "*"
    },
    {
      "Sid": "BootLogs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams",
        "logs:PutLogEvents",
        "logs:PutRetentionPolicy"
      ],
      "Resource": "*"
    },
    {
      "Sid": "StorageKeys",
      "Effect": "Allow",
      "Action": [
        "kms:CreateGrant",
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:GenerateDataKey",
        "kms:ReEncryptFrom",
        "kms:ReEncryptTo"
      ],
      "Resource": ["<your kms_key_id key ARN>"]
    }
  ]
}

The statements follow the same rules as the cluster role: Orchestration and BootLogs are not resource-scopable, and delete StorageKeys unless you set kms_key_id (same single key). Unlike the cluster role, the provisioner needs no S3 access: the cluster nodes perform all persistent-storage reads and writes.

TerraformAWS CLI

resource "aws_iam_role" "provisioner" {
  name               = "prodfs-qumulo-provisioner"
  assume_role_policy = file("trust.json")
}

resource "aws_iam_role_policy_attachment" "provisioner_ssm" {
  role       = aws_iam_role.provisioner.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
}

resource "aws_iam_role_policy_attachment" "provisioner_secrets" {
  role       = aws_iam_role.provisioner.name
  policy_arn = "arn:aws:iam::aws:policy/SecretsManagerReadWrite"
}

resource "aws_iam_role_policy" "provisioner" {
  role   = aws_iam_role.provisioner.name
  policy = file("provisioner-role-policy.json")
}

resource "aws_iam_instance_profile" "provisioner" {
  name = aws_iam_role.provisioner.name
  role = aws_iam_role.provisioner.name
}

aws iam create-role \
  --role-name prodfs-qumulo-provisioner \
  --assume-role-policy-document file://trust.json
aws iam attach-role-policy \
  --role-name prodfs-qumulo-provisioner \
  --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
aws iam attach-role-policy \
  --role-name prodfs-qumulo-provisioner \
  --policy-arn arn:aws:iam::aws:policy/SecretsManagerReadWrite
aws iam put-role-policy \
  --role-name prodfs-qumulo-provisioner \
  --policy-name provisioner-policy \
  --policy-document file://provisioner-role-policy.json

aws iam create-instance-profile \
  --instance-profile-name prodfs-qumulo-provisioner
aws iam add-role-to-instance-profile \
  --instance-profile-name prodfs-qumulo-provisioner \
  --role-name prodfs-qumulo-provisioner

What the deploying credentials need¶

The principal running terraform apply needs two grants on your role ARNs: iam:PassRole (every instance launch passes the role) and iam:ListInstanceProfilesForRole (the provider resolves each role to its instance profile with one read call).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "PassQumuloRoles",
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": [
        "arn:aws:iam::123456789012:role/prodfs-qumulo-nodes",
        "arn:aws:iam::123456789012:role/prodfs-qumulo-provisioner"
      ],
      "Condition": { "StringEquals": { "iam:PassedToService": "ec2.amazonaws.com" } }
    },
    {
      "Sid": "ResolveInstanceProfiles",
      "Effect": "Allow",
      "Action": "iam:ListInstanceProfilesForRole",
      "Resource": [
        "arn:aws:iam::123456789012:role/prodfs-qumulo-nodes",
        "arn:aws:iam::123456789012:role/prodfs-qumulo-provisioner"
      ]
    }
  ]
}

If you also set audit_logging = true, the deploying credentials need one more statement to manage the CloudWatch audit log group, independent of which roles you bring:

{
  "Sid": "AuditLogGroup",
  "Effect": "Allow",
  "Action": [
    "logs:DescribeLogGroups",
    "logs:CreateLogGroup",
    "logs:TagResource"
  ],
  "Resource": "*"
}

logs:DeleteLogGroup is deliberately not included -- the audit log group is designed to outlive the cluster, and the provider never deletes it. Omit this statement entirely if you leave audit_logging at its default false.

Note

permissions_boundary_arn applies only to roles the provider creates. With both roles brought, it has no effect.

Complete example¶

A runnable configuration that creates both roles, instance profiles, and policies, then deploys a cluster with them, lives in the provider repo under examples/aws-byo-iam/.

Rules¶

The roles must outlive the deployment. Scaling, node replacement, and configuration updates re-resolve them. Delete them only after terraform destroy.
Exactly one instance profile per role. Roles created in the AWS console get a same-named profile automatically; in Terraform, create aws_iam_instance_profile explicitly.
Import: both attributes are customer-declared. When importing a deployment that uses BYO roles, set them in config before terraform import; the provider does not discover them from AWS.

If something goes wrong¶

Symptom	Cause	Fix
Apply fails: role "is not in any instance profile"	The role was created without a profile	Create a profile containing the role (see Create the roles)
Apply fails: role "is in N instance profiles"	The role is in several profiles, so there is no unambiguous choice	Keep the role in exactly one profile
Instance launch fails with an authorization error	Deploying credentials lack `iam:PassRole` on the role	Add the PassRole statement above
Cluster deploys fine, but a later storage scale-up fails on S3 access	The new buckets fall outside your S3 wildcard, usually a `deployment_name` mismatch between the policy and the resource block	Make the embedded name match exactly, then re-apply