Skip to content

Put a Network Load Balancer in front of a multi-AZ AWS cluster

When you finish, clients reach a six-node cluster across three availability zones through one internal DNS name that keeps answering when a zone goes down.

Before you start

  • Three subnets in one VPC, each in a different availability zone, that meet the AWS network prerequisites. The cluster and the load balancer share them.
  • AWS credentials with the AWS IAM permissions the provider needs, plus permission to create Elastic Load Balancing resources
  • Client CIDRs for allow_cidrs that also cover the three subnets, because the load balancer passes client source IPs through and sends health checks from its own subnet addresses
  • Terraform 1.11 or later

1. Copy the configuration

Save this as main.tf in an empty directory.

terraform {
  required_version = ">= 1.11"
  required_providers {
    qumulo = {
      source  = "qumulo-terraform-registry.s3.us-east-1.amazonaws.com/qumulo/qumulo"
      version = "~> 1.0"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

locals {
  region = "us-east-1"             # <-- Replace with your region
  vpc_id = "vpc-0123456789abcdef0" # <-- Replace with your VPC ID
  subnet_ids = [
    "subnet-0aaaaaaaaaaaaaaaa", # <-- Replace with a subnet in your first availability zone
    "subnet-0bbbbbbbbbbbbbbbb", # <-- Replace with a subnet in your second availability zone
    "subnet-0cccccccccccccccc", # <-- Replace with a subnet in your third availability zone
  ]
  node_count = 6

  # Target group names cannot contain underscores, so the keys use hyphens.
  qumulo_ports = {
    ssh         = { port = 22, protocol = "TCP" }
    nfs-portmap = { port = 111, protocol = "TCP_UDP" }
    https       = { port = 443, protocol = "TCP" }
    smb         = { port = 445, protocol = "TCP" }
    nfs         = { port = 2049, protocol = "TCP_UDP" }
    api         = { port = 8000, protocol = "TCP" }
    s3          = { port = 9000, protocol = "TCP" }
  }

  # Keyed on node_count rather than primary_ips, which is unknown until the
  # cluster exists and would make for_each fail on the first plan.
  qumulo_attachments = merge([
    for port_key, _ in local.qumulo_ports : {
      for node_idx in range(local.node_count) :
      "${port_key}-${node_idx}" => { port_key = port_key, node_idx = node_idx }
    }
  ]...)
}

provider "qumulo" {
  aws {}
}

provider "aws" {
  region = local.region
}

resource "qumulo_filesystem_aws" "cluster" {
  cluster_name         = "qumulo-nlb"
  deployment_name      = "qumulo-nlb"
  region               = local.region
  subnet_ids           = local.subnet_ids
  node_count           = local.node_count
  instance_type        = "m6idn.4xlarge"
  cluster_product_type = "HOT"
  admin_password       = "YourSecurePassword123!" # <-- Replace with your admin password
  allow_cidrs          = ["10.0.0.0/8"]           # <-- Replace with your client networks
  deletion_protection  = true
}

resource "aws_lb" "qumulo" {
  name                             = "qumulo-nlb"
  internal                         = true
  load_balancer_type               = "network"
  subnets                          = local.subnet_ids
  enable_cross_zone_load_balancing = true
}

resource "aws_lb_target_group" "qumulo" {
  for_each = local.qumulo_ports

  name               = "qumulo-${each.key}"
  port               = each.value.port
  protocol           = each.value.protocol
  target_type        = "ip"
  vpc_id             = local.vpc_id
  preserve_client_ip = true

  # AWS rejects source_ip stickiness on TCP_UDP target groups.
  dynamic "stickiness" {
    for_each = each.value.protocol == "TCP_UDP" ? [] : [1]
    content {
      enabled = true
      type    = "source_ip"
    }
  }
}

resource "aws_lb_target_group_attachment" "qumulo" {
  for_each = local.qumulo_attachments

  target_group_arn = aws_lb_target_group.qumulo[each.value.port_key].arn
  target_id        = qumulo_filesystem_aws.cluster.primary_ips[each.value.node_idx]
  port             = local.qumulo_ports[each.value.port_key].port
}

resource "aws_lb_listener" "qumulo" {
  for_each = local.qumulo_ports

  load_balancer_arn = aws_lb.qumulo.arn
  port              = each.value.port
  protocol          = each.value.protocol

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.qumulo[each.key].arn
  }
}

output "nlb_dns" {
  value = aws_lb.qumulo.dns_name
}

output "web_ui" {
  value = "https://${aws_lb.qumulo.dns_name}"
}

output "https_target_group_arn" {
  value = aws_lb_target_group.qumulo["https"].arn
}

Replace the seven values marked <-- Replace, and leave the rest as written. To stop exposing a port, delete its entry from qumulo_ports. To front a cluster you already manage, keep your own qumulo_filesystem_aws block and point the attachments at its primary_ips.

Three settings are deliberate and worth leaving on. Cross-zone load balancing is what lets a client fail over when a zone goes down, and AWS bills inter-AZ transfer for it. Source-IP stickiness keeps SMB multichannel and NFSv4 state on one node, so a fleet behind a single NAT address lands on one node. Client IP preservation keeps real client addresses in Qumulo audit logs, which is why allow_cidrs must cover the client networks rather than the load balancer.

2. Apply

terraform init -upgrade
terraform plan
terraform apply

The plan lists the cluster, the load balancer, seven target groups with their listeners, and one attachment per node per target group. The apply typically runs 60 to 80 minutes and ends with Apply complete!, followed by the https_target_group_arn, nlb_dns, and web_ui outputs.

3. Check the cluster

Ask the load balancer whether every node passes its health check on the HTTPS port. Replace us-east-1 with your region.

aws elbv2 describe-target-health \
  --region us-east-1 \
  --target-group-arn "$(terraform output -raw https_target_group_arn)" \
  --query 'TargetHealthDescriptions[].TargetHealth.State' \
  --output text

The command prints healthy six times, once per node. Open the web_ui address in a browser, accept the cluster's self-signed certificate, and sign in as admin with the password you set.

Warning

Mount NFS clients with NFSv4. NFSv3 locking is unreliable through a load balancer.

4. Give the load balancer a friendly name

If you want clients to use a name in your own private hosted zone, add a Route 53 alias record to main.tf and apply again.

data "aws_route53_zone" "internal" {
  name         = "internal.example.com" # <-- Replace with your private hosted zone
  private_zone = true
}

resource "aws_route53_record" "qumulo" {
  zone_id = data.aws_route53_zone.internal.zone_id
  name    = "qumulo.internal.example.com" # <-- Replace with the name clients will use
  type    = "A"

  alias {
    name                   = aws_lb.qumulo.dns_name
    zone_id                = aws_lb.qumulo.zone_id
    evaluate_target_health = true
  }
}

The apply ends with Apply complete! Resources: 1 added, 0 changed, 0 destroyed., and the new name resolves to the load balancer from inside the VPC.

Next

The qumulo_filesystem_aws reference lists every cluster attribute. If a step fails, see Troubleshoot an AWS deployment.