Skip to content

GCP Custom Images Example

This example demonstrates how to configure custom images for Qumulo cluster nodes and the provisioner VM on GCP. It covers:

  • the three ways each role selects an image
  • how to switch between them after a cluster exists
  • the replacement rules that apply when a node image changes

Prerequisites

  • GCP project with billing enabled
  • IAM permissions to create compute instances, GCS buckets, service accounts, and firewall rules
  • Existing VPC network and subnetwork in the target region, with Private Google Access enabled
  • (Optional) A custom image, or an image family, for cluster nodes and/or the provisioner VM
  • Terraform >= 1.0
  • Application Default Credentials (gcloud auth application-default login) or GOOGLE_APPLICATION_CREDENTIALS set

Project Prerequisites

Your GCP project must meet these requirements before deploying a cluster:

  • Enabled APIs: Compute Engine (compute.googleapis.com), Cloud Storage (storage.googleapis.com), and IAM (iam.googleapis.com) must be enabled. Cloud KMS (cloudkms.googleapis.com) is also required if kms_key_name is set.
  • IAM roles: The deploying principal needs roles/compute.admin, roles/storage.admin, and (when service_account_email is omitted) roles/iam.serviceAccountAdmin. Add roles/cloudkms.cryptoKeyEncrypterDecrypter on the KMS key if kms_key_name is set.
  • Image access: The deploying principal (and the cluster service account) must be able to read any custom image you reference. Grant roles/compute.imageUser on the image or its host project for cross-project images.
  • Network: An existing VPC and subnetwork in the target region. The subnet must have Private Google Access enabled so cluster nodes without external IPs can reach Cloud Storage.

See the GCS state backend example for a working IAM and project setup.

Usage

  1. Provide variable values. Create a terraform.tfvars file, use TF_VAR_* env vars, or pass -var / -var-file flags. At minimum:
project_id     = "your-gcp-project"
region         = "us-central1"
subnetwork     = "projects/your-gcp-project/regions/us-central1/subnetworks/qumulo-subnet"
admin_password = "YourSecurePassword123!"
allow_cidrs    = ["10.0.0.0/8"]
  1. Initialize and deploy: 
    terraform init -upgrade
terraform plan
terraform apply

Image Selection Options

Cluster nodes and the provisioner VM each select an image in one of three ways: the default curated image, an image family, or a pinned image. The two roles are independent: you can pin the node image while letting the provisioner take a family or the default, or any other combination.

Default (No Image Specified)

By default, the provider deploys the curated Ubuntu 24.04 LTS image (ubuntu-os-cloud / ubuntu-2404-lts-amd64) for both cluster nodes and the provisioner VM. Omit all image fields to get the default behavior.

Option 1: Image Family (latest in a family)

Set the (project, family) pair to resolve the latest image in that family at deploy time. This is the right choice when you want to track a maintained family (your own or a public one) rather than a fixed build:

node_image_project = "my-image-project"
node_image_family  = "my-hardened-ubuntu"

Each pair is all-or-nothing: setting node_image_project without node_image_family (or vice versa) is rejected at plan time. The same rule applies to the provisioner_image_project / provisioner_image_family pair.

Option 2: Pinned Image (exact build)

Pin an exact image with the partial-form path projects/PROJECT/global/images/IMAGE. A pin takes precedence over the (project, family) pair for that role, so you can leave the pair set and still deploy the pin:

node_image        = "projects/my-image-project/global/images/qumulo-7-1-2-3"
provisioner_image = "projects/my-image-project/global/images/qumulo-provisioner-7-1"

Option 3: Mixed Across Roles

Pin the cluster nodes to an exact build while letting the provisioner track a family (or vice versa):

node_image                = "projects/my-image-project/global/images/qumulo-7-1-2-3"
provisioner_image_project = "my-image-project"
provisioner_image_family  = "qumulo-provisioner"

Switching Modes on an Existing Cluster

The image attributes are Computed: deleting a line from your configuration does not clear the value. The provider keeps the last-applied value. To move a role off a pin or a custom family, set the field to an empty string (""):

From To Configuration
Pinned image Image family node_image = "" and set node_image_project + node_image_family
Image family Default Ubuntu node_image_project = "" and node_image_family = ""
Pinned image Default Ubuntu node_image = "" and leave the project/family pair empty

The same ""-to-clear pattern applies to every provisioner_image* field.

Replacement Behavior

  • Changing any node image field (node_image, node_image_project, or node_image_family) replaces the cluster: Terraform destroys the existing cluster and creates a new one. Plan for data migration and downtime, and take a snapshot or replicate off-cluster first.
  • Changing any provisioner image field does NOT replace the cluster. The provisioner VM is ephemeral, so the new image is used on the next deploy operation (for example, the next scale or version change).

Outputs

After deployment, you'll get:

  • cluster_name: The name of your cluster
  • cluster_uuid: UUID of the Qumulo cluster
  • endpoint_ips: IP addresses for client connections (floating IPs if configured, otherwise primary IPs)
  • primary_ips: Per-node primary IPs
  • endpoints: Pre-formatted connection strings for web UI, API, NFS, and SMB

Full Configuration

# Example: GCP Qumulo Cluster with Custom Images
# Edit the values below directly, then run: terraform init -upgrade && terraform apply

terraform {
  required_version = ">= 1.0"
  required_providers {
    qumulo = {
      source  = "qumulo-terraform-registry.s3.us-east-1.amazonaws.com/qumulo/qumulo"
      version = "~> 1.0"
    }
  }
}

provider "qumulo" {
  gcp {
    # project_id and region can also come from the GOOGLE_PROJECT and
    # GOOGLE_REGION environment variables. Application Default Credentials
    # (gcloud auth application-default login) provide authentication.
    project_id = var.project_id
    region     = var.region
  }
}

variable "project_id" {
  description = "GCP project ID for deployment"
  type        = string
}

variable "region" {
  description = "GCP region for deployment"
  type        = string
  default     = "us-central1"
}

variable "subnetwork" {
  description = "Subnetwork name or self-link. Must have Private Google Access enabled."
  type        = string
}

variable "admin_password" {
  description = "Cluster administrator password (8-128 chars)"
  type        = string
  sensitive   = true
  validation {
    condition     = length(var.admin_password) >= 8 && length(var.admin_password) <= 128
    error_message = "admin_password must be between 8 and 128 characters."
  }
}

variable "allow_cidrs" {
  description = "CIDR blocks allowed to access the cluster"
  type        = list(string)
  default     = ["0.0.0.0/0"]
}

resource "qumulo_filesystem_gcp" "cluster" {
  provider = qumulo

  # =============================================================================
  # All attributes (alphabetical order).
  # Uncomment any attribute to use it. Required attributes are not commented.
  # =============================================================================

  # Administrator password for cluster access (8-128 characters).
  admin_password = var.admin_password

  # CIDR blocks allowed to access the cluster via firewall rules (required, at least one).
  allow_cidrs = var.allow_cidrs

  # GCE machine type for cluster nodes.
  instance_type = "n2-highmem-8"

  # qfsd cluster name, shown in the Qumulo UI (2-15 chars, case preserved).
  cluster_name    = "qumuloimg"

  # Prefix for the GCP resources this cluster creates (2-15 lowercase chars).
  deployment_name = "qumuloimg"

  # --- Cluster node image ------------------------------------------------------
  # Choose ONE mode. Omit all three for the default Ubuntu 24.04 image.
  #
  # Pinned image (exact build; takes precedence over the project/family pair):
  node_image = "projects/my-image-project/global/images/qumulo-7-1-2-3"
  #
  # Image family (latest in family; all-or-nothing pair):
  # node_image_project = "my-image-project"
  # node_image_family  = "my-hardened-ubuntu"
  #
  # To switch an existing cluster OFF a pin, set node_image = "" and supply the
  # pair. To return to the default, clear both the pin and the pair with "".
  # Changing any node image field REPLACES the cluster (destroy + recreate).

  # Number of nodes in the cluster.
  node_count = 3

  # GCP project ID. Defaults to the provider gcp { project_id } value.
  project_id = var.project_id

  # --- Provisioner VM image ----------------------------------------------------
  # Ephemeral; changes do NOT replace the cluster. Same three modes as the node.
  provisioner_image = "projects/my-image-project/global/images/qumulo-provisioner-7-1"
  # provisioner_image_project = "my-image-project"
  # provisioner_image_family  = "qumulo-provisioner"

  # GCP region. Defaults to the provider gcp { region } value.
  region = var.region

  # Usable storage capacity in TB.
  soft_capacity_limit_tb = 100

  # Subnetwork name or self-link. Must have Private Google Access enabled.
  subnetwork = var.subnetwork

  deletion_protection = true # recommended: guard the cluster's node instances

  timeouts {
    create = "90m"
    delete = "90m"
  }
}

output "cluster_name" {
  description = "Name of the Qumulo cluster"
  value       = qumulo_filesystem_gcp.cluster.cluster_name
}

output "cluster_uuid" {
  description = "UUID of the Qumulo cluster"
  value       = qumulo_filesystem_gcp.cluster.cluster_uuid
}

output "endpoint_ips" {
  description = "Client-facing IPs. Floating IPs if configured, otherwise primary IPs."
  value       = qumulo_filesystem_gcp.cluster.endpoint_ips
}

output "primary_ips" {
  description = "Per-node primary IPs. Use these directly when no floating IPs are configured, or for per-node access."
  value       = qumulo_filesystem_gcp.cluster.primary_ips
}

output "endpoints" {
  description = "Connection endpoints for various protocols"
  value = {
    web_ui = "https://${try(qumulo_filesystem_gcp.cluster.endpoint_ips[0], "pending")}"
    api    = "https://${try(qumulo_filesystem_gcp.cluster.endpoint_ips[0], "pending")}:8000"
    nfs    = "${try(qumulo_filesystem_gcp.cluster.endpoint_ips[0], "pending")}:/"
    smb    = "\\\\${try(qumulo_filesystem_gcp.cluster.endpoint_ips[0], "pending")}\\share"
  }
}