Production AWS Qumulo Cluster Example¶

This example demonstrates a production-ready AWS deployment with multi-AZ high availability, customer-managed KMS encryption, IAM permissions boundary, and enterprise security controls.

Prerequisites¶

AWS account with permissions to create IAM roles, EC2 instances, S3 buckets, KMS keys, and security groups (or equivalent permissions boundary)
ec2:ModifyInstanceAttribute and s3:GetBucketPolicy on the deploying credentials, for every cluster: the provider lifts deletion protection from an instance or bucket immediately before terminating or deleting it, whether or not deletion_protection was ever set
Using deletion_protection additionally requires ec2:DescribeInstanceAttribute, so refresh can report the live protection state (s3:PutBucketPolicy is already required)
Using audit_logging additionally requires logs:DescribeLogGroups, logs:CreateLogGroup, and logs:TagResource on the deploying credentials (logs:DeleteLogGroup is deliberately not required -- the audit log group is designed to outlive the cluster, and the provider never deletes it)
Existing VPC and 3+ subnets, one per availability zone
(Optional) pre-created KMS customer-managed key (CMK) for cluster data
(Optional) pre-created IAM permissions boundary policy
(Optional) SSH EC2 key pair, or use AWS Systems Manager Session Manager with an IAM permissions boundary that allows it
Terraform >= 1.0

VPC Prerequisites

Your VPC must meet two requirements before deploying a cluster:

S3 gateway VPC endpoint: Attach an S3 gateway endpoint to the route table serving the subnet. The provider does not create this (cross-account VPCs may lack permission), and deployment will fail without it.
Outbound internet access: Cluster nodes and the provisioner instance must reach the public internet for image downloads and cluster registration. Use a NAT Gateway in a private subnet (recommended). An Internet Gateway in a public subnet also works but reduces network isolation.

Deploying into a VPC owned by another AWS account?

For RAM-shared subnets (where the VPC lives in a different AWS account than the one running Terraform), see AWS Cross-Account VPC for the additional setup and the cluster_security_group_id / provisioner_security_group_id Bring-your-own pattern.

Usage¶

Provide variable values. Create a terraform.tfvars file (or use TF_VAR_* env vars, or pass -var / -var-file flags):

region                   = "us-east-1"
vpc_id                   = "vpc-0123456789abcdef0"
subnet_ids               = ["subnet-aaa", "subnet-bbb", "subnet-ccc"]
admin_password           = "YourSecurePassword123!"
allow_cidrs              = ["10.0.0.0/8"]
ec2_key_pair             = "my-prod-keypair"
nexus_registration_key   = "your-nexus-key"
permissions_boundary_arn = "arn:aws:iam::123456789012:policy/production-boundary"

Edit the KMS alias in main.tf (look for the <-- Replace comment in the data "aws_kms_key" block).

Initialize and deploy:

terraform init -upgrade
terraform plan
terraform apply

Features¶

This example demonstrates:

Multi-AZ deployment across 3 subnets in distinct availability zones
KMS encryption for cluster data using a customer-managed CMK
IAM permissions boundary applied to cluster and provisioner roles
Nexus registration for remote support
EC2 key pair for SSH access
Production-appropriate tagging
cluster_product_type chooses between HOT (default, optimized for frequently accessed data) and COLD (optimized for archival). This is immutable after creation, so pick deliberately for production.
storage_class optionally selects the S3 storage class backing persistent storage. HOT clusters support STANDARD and INTELLIGENT_TIERING (default); COLD clusters support STANDARD_IA and GLACIER_IR (default). Also immutable after creation.

Optional Features¶

The full configuration below includes commented-out blocks for features you can enable as needed:

Additional security groups: Attach pre-existing SGs for custom ingress rules beyond what the provider creates
Custom AMIs: Override the default cluster or provisioner AMIs (see AWS Custom Images for details)
Floating IPs: Single-AZ only, not compatible with this multi-AZ example. For a stable client endpoint across availability zones, see AWS Multi-AZ NLB.
Networking mode override: Only change when instructed by Qumulo support
Custom cluster version: Pin to a specific Qumulo software release
Custom provisioner instance type: Override the default provisioner VM size

Outputs¶

cluster_name: Name of the Qumulo cluster
cluster_uuid: UUID of the Qumulo cluster
deployment_unique_name: Unique deployment identifier
endpoint_ips: Client-facing IPs. Floating IPs if configured, otherwise primary IPs.
primary_ips: Per-node primary IPs. Use these directly when no floating IPs are configured, or for per-node access.
endpoints: Connection endpoints for various protocols (web UI, API, NFS, SMB)

Multi-AZ Sizing¶

A valid deployment uses either 1 subnet (single-AZ) or 3+ subnets (multi-AZ). 2 subnets is never valid because it cannot form a majority quorum.

For multi-AZ:

Supply 3 or more subnet IDs, each in a distinct availability zone. The provider calls DescribeSubnets to derive each subnet's AZ, so mixing two subnets from the same AZ will fail validation.
Floating IPs (floating_ip_count) require single-AZ and cannot be used with multi-AZ deployments.

For standard production workloads, use 3 subnets across 3 AZs and scale node_count and soft_capacity_limit_tb to your capacity and throughput targets.

KMS Encryption¶

Pass a pre-created customer-managed key to encrypt cluster data:

terraform {
  required_providers {
    qumulo = {
      source  = "qumulo-terraform-registry.s3.us-east-1.amazonaws.com/qumulo/qumulo"
      version = "~> 1.0"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "qumulo" {
  aws {}
}

provider "aws" {
  region = "us-east-1"
}

data "aws_kms_key" "cluster" {
  key_id = "alias/qumulo-cluster"
}

resource "qumulo_filesystem_aws" "cluster" {
  # ... required fields ...
  kms_key_id          = data.aws_kms_key.cluster.arn
  deletion_protection = true # recommended: guard the cluster's EC2 instances and S3 buckets

  timeouts {
    create = "90m"
    delete = "30m"
  }
}

kms_key_id is immutable after creation. The key must live in the same region as the cluster, and the IAM roles the provider creates must be able to kms:Encrypt, kms:Decrypt, kms:GenerateDataKey, and kms:DescribeKey against it.

Permissions Boundary¶

Attach a pre-created IAM permissions boundary ARN to restrict the policies the provider can attach to the cluster and provisioner IAM roles:

terraform {
  required_providers {
    qumulo = {
      source  = "qumulo-terraform-registry.s3.us-east-1.amazonaws.com/qumulo/qumulo"
      version = "~> 1.0"
    }
  }
}

provider "qumulo" {
  aws {}
}

resource "qumulo_filesystem_aws" "cluster" {
  # ... required fields ...
  permissions_boundary_arn = "arn:aws:iam::123456789012:policy/qumulo-boundary"
  deletion_protection      = true # recommended: guard the cluster's EC2 instances and S3 buckets

  timeouts {
    create = "90m"
    delete = "30m"
  }
}

The provider creates two IAM roles, one for cluster nodes and one for the provisioner, and attaches the same boundary to both. The boundary must permit the permissions required by the cluster node and provisioner roles (S3, EC2, KMS, Secrets Manager, CloudWatch Logs). If the boundary forbids any required permission, cluster role creation will fail with an IAM error.

Troubleshooting¶

S3 gateway endpoint missing: Cluster creation fails early if the subnet's route table has no S3 gateway endpoint. Add one in the VPC console under Endpoints > Create endpoint > com.amazonaws..s3 (type: Gateway), and associate it with every route table used by your cluster subnets.
Outbound unreachable: If any subnet has no internet route, the provisioner cannot download the installer. Confirm a 0.0.0.0/0 route to a NAT Gateway (preferred) or Internet Gateway for each subnet in subnet_ids.
Subnets not in distinct AZs: Multi-AZ deployments require each subnet to be in a different availability zone. The provider calls DescribeSubnets to derive AZs and rejects configurations with duplicate zones. Verify each subnet's AZ in the VPC console before applying.
KMS key not accessible from the region: A customer-managed CMK is regional. Passing a kms_key_id from a different region than region will fail. Create the CMK in the same region, or use a multi-region key replica.
Permissions boundary too restrictive: If the boundary forbids any permission the cluster or provisioner role needs, iam:CreateRole or iam:AttachRolePolicy will fail. Review the boundary against the permissions the provider requests and widen it (or pick a less restrictive boundary).
Nexus registration key invalid or expired: Nexus keys are single-use and time-bound. Generate a fresh key at https://nexus.qumulo.com/user/registration-key immediately before running terraform apply.
Debug logs: Run TF_LOG=DEBUG terraform apply for detailed output.

Full Configuration¶

# Example: Production AWS Qumulo Cluster (multi-AZ)
# Edit the values below directly, then run: terraform init -upgrade && terraform apply

terraform {
  required_version = ">= 1.0"
  required_providers {
    qumulo = {
      source  = "qumulo-terraform-registry.s3.us-east-1.amazonaws.com/qumulo/qumulo"
      version = "~> 1.0"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "qumulo" {
  aws {
    # Standard AWS credential chain applies (env vars, shared config, SSO,
    # instance profile). No credentials block is required here.
    # profile   = "my-profile"   # optional, overrides AWS_PROFILE
  }
}

provider "aws" {
  region = var.region
  # Credentials come from the standard AWS credential chain
  # (env vars, shared config, SSO, instance profile).
}

variable "region" {
  description = "AWS region for deployment"
  type        = string
  default     = "us-east-1"
}

variable "vpc_id" {
  description = "VPC ID for the cluster"
  type        = string
}

variable "subnet_ids" {
  description = "Subnet IDs for multi-AZ deployment (3+ subnets, one per AZ)"
  type        = list(string)
}

variable "admin_password" {
  description = "Administrator password for cluster access"
  type        = string
  sensitive   = true
  validation {
    condition     = length(var.admin_password) >= 8 && length(var.admin_password) <= 128
    error_message = "admin_password must be 8-128 characters."
  }
}

variable "allow_cidrs" {
  description = "CIDR blocks allowed to access the cluster"
  type        = list(string)
}

variable "ec2_key_pair" {
  description = "EC2 key pair name for SSH access to cluster nodes"
  type        = string
}

variable "nexus_registration_key" {
  description = "Qumulo Nexus registration key for remote support"
  type        = string
  sensitive   = true
}

variable "permissions_boundary_arn" {
  description = "IAM permissions boundary ARN applied to cluster and provisioner roles"
  type        = string
  default     = ""
}

# Pre-created customer-managed KMS key for cluster data encryption.
# Replace the alias with your own, or switch to a key_id / ARN lookup.
data "aws_kms_key" "cluster" {
  key_id = "alias/qumulo-cluster" # <-- Replace with your KMS alias
}

resource "qumulo_filesystem_aws" "cluster" {
  provider = qumulo

  # =============================================================================
  # All attributes (alphabetical order).
  # Uncomment any attribute to use it. Required attributes are not commented.
  # =============================================================================

  # # Additional security group IDs to attach to cluster nodes.
  # # Use for adding custom ingress rules beyond those the provider creates.
  # additional_security_group_ids = ["sg-abc123", "sg-def456"]

  # Administrator password for cluster access (8-128 characters).
  # Sensitive and write-only -- not stored in Terraform state.
  admin_password = var.admin_password

  # CIDR blocks allowed to access the cluster (required, at least one).
  # Production clusters should restrict to known client/management networks.
  allow_cidrs = var.allow_cidrs

  # # AMI ID for cluster nodes. If omitted, the default Qumulo AMI (Ubuntu 24.04) is used.
  # # Supports Ubuntu and RHEL 8, 9, and 10 AMIs. See aws-custom-images.md for details.
  # ami_id = "ami-0123456789abcdef0"

  # Cluster storage product type (immutable after creation).
  # HOT: Optimized for frequently accessed data.
  # COLD: Optimized for archival/infrequently accessed data.
  cluster_product_type = "HOT"

  # # S3 storage class backing persistent storage (immutable after creation).
  # # HOT: STANDARD or INTELLIGENT_TIERING (default).
  # # COLD: STANDARD_IA or GLACIER_IR (default).
  # storage_class = "STANDARD"

  # # Qumulo software version. Defaults to latest. Immutable after creation.
  # cluster_version = "7.5.0"

  # EC2 key pair name for SSH access to cluster nodes.
  ec2_key_pair = var.ec2_key_pair

  # # Number of floating IPs. Must be 0, or between 3 and 100.
  # # Requires networking_mode = "host_managed" AND single-AZ deployment.
  # # NOT compatible with this multi-AZ example.
  # floating_ip_count = 3

  # EC2 instance type for cluster nodes.
  # Production sizing typically starts at m6idn.4xlarge and scales up.
  # Supported families include m6idn, m6i, m7i, i3en, i4i, i7i, i7ie. See the basic example's
  # "Supported Instance Types" table for the full list.
  instance_type = "m6idn.4xlarge"

  # KMS key ARN for encrypting cluster data (immutable after creation).
  # Must be in the same region as the cluster.
  kms_key_id = data.aws_kms_key.cluster.arn

  # qfsd cluster name, shown in the Qumulo UI (2-15 chars, case preserved).
  cluster_name = "qumulo-prod"

  # Prefix for the AWS resources this cluster creates (2-15 lowercase chars).
  deployment_name = "qumulo-prod"

  # # Networking mode. Default "host_managed" preserves EC2 IPs across reboots.
  # # Use "qumulo_managed" only when instructed by Qumulo support.
  # networking_mode = "host_managed"

  # Qumulo Nexus registration key for remote support.
  # Obtain from https://nexus.qumulo.com/user/registration-key
  nexus_registration_key = var.nexus_registration_key

  # Number of nodes in the cluster.
  # Production deployments typically start at 6 nodes and scale from there.
  # Valid values: 1 (single node), or 3-24 (4 nodes: single-AZ only).
  node_count = 6

  # IAM permissions boundary ARN applied to the cluster and provisioner roles.
  # The provider attaches the same boundary to both roles it creates.
  permissions_boundary_arn = var.permissions_boundary_arn

  # # AMI ID for the provisioner instance. Defaults to the Qumulo provisioner AMI.
  # # See aws-custom-images.md for details.
  # provisioner_ami_id = "ami-0123456789abcdef0"

  # # EC2 instance type for the provisioner VM (used during deploy operations).
  # # Default: m5.xlarge. Set a larger type if you hit provisioner CPU/memory limits during deployment.
  # provisioner_instance_type = "m5.xlarge"

  # AWS region for deployment.
  region = var.region

  # Soft capacity limit in TB (50 to 50000).
  # Can be increased to add storage, but cannot be decreased.
  soft_capacity_limit_tb = 5000

  # Subnet IDs for cluster nodes.
  # Must be 3+ subnets for multi-AZ, each in a distinct availability zone.
  subnet_ids = var.subnet_ids

  # Tags to apply to all AWS resources created for this cluster.
  tags = {
    Environment        = "Production"
    ManagedBy          = "Terraform"
    Service            = "Qumulo"
    CostCenter         = "Storage"
    DataClassification = "Confidential"
    BackupPolicy       = "Daily"
    SLA                = "99.99"
  }

  # VPC ID. Set explicitly in production rather than relying on derivation.
  vpc_id = var.vpc_id

  deletion_protection = true # recommended: guard the cluster's EC2 instances and S3 buckets

  timeouts {
    create = "90m"
    delete = "30m"
  }
}

output "cluster_name" {
  description = "Name of the Qumulo cluster"
  value       = qumulo_filesystem_aws.cluster.cluster_name
}

output "cluster_uuid" {
  description = "UUID of the Qumulo cluster"
  value       = qumulo_filesystem_aws.cluster.cluster_uuid
}

output "deployment_unique_name" {
  description = "Unique deployment identifier"
  value       = qumulo_filesystem_aws.cluster.deployment_unique_name
}

output "endpoint_ips" {
  description = "Client-facing IPs. Floating IPs if configured, otherwise primary IPs."
  value       = qumulo_filesystem_aws.cluster.endpoint_ips
}

output "primary_ips" {
  description = "Per-node primary IPs. Use these directly when no floating IPs are configured, or for per-node access."
  value       = qumulo_filesystem_aws.cluster.primary_ips
}

output "endpoints" {
  description = "Connection endpoints for various protocols"
  value = {
    web_ui = "https://${try(qumulo_filesystem_aws.cluster.endpoint_ips[0], "pending")}"
    api    = "https://${try(qumulo_filesystem_aws.cluster.endpoint_ips[0], "pending")}:8000"
    nfs    = "${try(qumulo_filesystem_aws.cluster.endpoint_ips[0], "pending")}:/"
    smb    = "\\\\${try(qumulo_filesystem_aws.cluster.endpoint_ips[0], "pending")}\\share"
  }
}