Terraform on GCP: repeatable environments for client projects

By Marco Lopes · May 23, 2022 · Cloud & DevOps

At Kordspace, we work remotely across software, AI, Web3, hardware, and design. That breadth makes a consistent infrastructure workflow important: a prototype might begin as an API and later need data processing, device ingestion, or a separate environment for a client’s internal team.

We prefer Google Cloud for many of these projects, while also working with AWS and Azure. Regardless of provider, we want infrastructure we can explain, review, recreate, and hand over without relying on someone’s memory of the cloud console.

Terraform gives us a practical foundation for that. It does not remove the need for architecture decisions or operational discipline. It makes those decisions explicit and gives us a repeatable way to apply them.

Here is how we approach Terraform on GCP for client projects.

Start with boundaries, not resources

Before writing a resource block, we agree on ownership. Who owns the Google Cloud organization, billing account, domain, repository, and deployment credentials? Who approves production changes? Who will operate the system after handover?

Where possible, we place infrastructure in client-owned accounts from the beginning. Moving ownership later can be more complicated than moving application code, especially when billing, identity, and organization policies are involved.

We normally separate development, staging, and production into distinct GCP projects. Projects provide useful boundaries for IAM, quotas, billing visibility, and enabled APIs. They are not a substitute for careful permissions, but they reduce accidental overlap between environments.

We also agree on region, data location, naming, and labels early. A short decision record is enough. The goal is to prevent every new resource from becoming a fresh debate.

Make environments similar, not necessarily identical

Repeatability does not mean paying for three production-sized installations.

We want environments to share the same architectural pattern: the same module interfaces, service relationships, security assumptions, and deployment process. We can still give development smaller instances, lower limits, or fewer availability features.

We make those differences explicit through inputs rather than hidden console changes. If production needs high availability and development does not, we record that distinction in configuration and review its consequences.

Some differences deserve extra attention. A staging database with different extensions, a different authentication path, or a different network model may fail to catch production problems. We prioritize similarity where it affects behavior, not merely where it makes diagrams look consistent.

Keep the repository easy to navigate

For a small project, we favor a simple structure with reusable modules and separate root configurations:

infra/
  modules/
    application/
    database/
  environments/
    dev/
      main.tf
      variables.tf
      outputs.tf
      backend.tf
    staging/
    production/

Each environment gets its own backend configuration and state. We commit the dependency lock file for each root configuration so collaborators and CI resolve the same provider selections.

We do not begin with a universal module that can deploy every conceivable application. We extract modules when they represent a useful, understandable boundary: an application service and its identity, for example, or a database with its supporting configuration.

Terraform workspaces can be useful, but we do not treat workspace names as a security boundary. Separate directories, states, projects, and credentials make client environment boundaries easier to inspect.

Pin versions and review upgrades

A repeatable configuration depends on more than its resource definitions. Terraform and provider versions matter too.

For a project starting today, a baseline could look like this:

terraform {
  required_version = "~> 1.1.0"

  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 4.0"
    }
  }
}

provider "google" {
  project = var.project_id
  region  = var.region
}

These constraints are an example, not a requirement for every project. We choose versions against the resources we need, then use the lock file to preserve the selected provider version and checksums.

We make upgrades deliberate changes. We read release notes, update in a branch, inspect the resulting plan, and test in a lower-risk environment before production. An unrelated application change should not quietly introduce a provider upgrade.

Treat remote state as sensitive infrastructure

Terraform state records the relationship between configuration and real resources. For a remote team, keeping the only copy on a laptop is not a workable collaboration model.

We typically use a dedicated Google Cloud Storage bucket with separate state locations for each environment:

terraform {
  backend "gcs" {
    bucket = "example-client-terraform-state"
    prefix = "environments/dev"
  }
}

The bucket name is illustrative and must be replaced with an available, project-specific name. Backend configuration cannot use ordinary Terraform input variables; we either keep non-secret settings here or supply them during initialization.

We enable object versioning, restrict access, and use uniform bucket-level access. We treat state as sensitive because it can contain credentials or other confidential resource attributes. Marking an output sensitive hides it in some displays; it does not remove it from state.

The GCS backend supports state locking. That helps prevent concurrent writes to the same state, but it does not coordinate changes across separate states. We still serialize applies per environment in CI.

Solve the bootstrap problem explicitly

The state bucket must exist before Terraform can use it as a backend. Project creation, billing attachment, initial IAM, and API enablement can introduce similar dependencies.

We document a small bootstrap procedure rather than pretending these steps happen automatically. Depending on client access, we either provision the foundation through a separate Terraform root or perform a tightly controlled initial setup and import resources where appropriate.

If we begin with local state, we migrate it to the remote backend with terraform init -migrate-state after the bucket is ready. We verify the migration before removing local copies and keep all state files out of Git.

We also separate foundation permissions from everyday deployment permissions. Creating projects and changing billing should not be necessary for routine application infrastructure updates.

Make APIs and identities part of the design

GCP services often require an API to be enabled before their resources can be created. We manage that dependency explicitly:

resource "google_project_service" "cloud_run" {
  project            = var.project_id
  service            = "run.googleapis.com"
  disable_on_destroy = false
}

We use disable_on_destroy = false when removing this Terraform resource should not disable an API that other resources may still use. We assign one clear configuration owner for API enablement rather than having several states manage the same setting.

For identities, we distinguish the Terraform deployment identity from application runtime service accounts. The application should not inherit infrastructure administration privileges simply because that is convenient during setup.

We grant permissions for defined tasks and scopes. We also take care with IAM resource semantics: an authoritative policy or role binding can remove access managed elsewhere. Additive member resources can be safer when ownership is shared, provided we avoid conflicting management of the same grants.

Build a reviewable plan-and-apply workflow

Our infrastructure workflow follows the same principle as our software workflow: changes belong in version control and deserve review.

A pull request runs formatting and validation:

terraform fmt -check -recursive
terraform init -input=false
terraform validate
terraform plan -out=tfplan

In practice, we run these from the appropriate directories, with planning authenticated against the intended environment. Validation catches structural problems; a plan shows proposed infrastructure changes. Neither proves that the resulting system will behave correctly.

We review replacements and deletions carefully, especially for databases, storage, network dependencies, and identities. After approval, we apply the saved plan through a controlled job. If configuration or infrastructure changes invalidate it, we create and review a fresh plan.

Saved plans can contain sensitive data, so we protect them as short-lived artifacts. We do not expose production credentials to untrusted pull requests, and we prefer short-lived authentication, such as workload identity federation where supported, over stored service-account keys.

Separate infrastructure delivery from application delivery

Terraform can configure an application platform without becoming responsible for every application release.

We decide explicitly which workflow owns a deployed image, revision, or service configuration. If Terraform and an application pipeline both manage the same field, they can repeatedly overwrite each other’s changes.

For some small projects, managing the image reference through Terraform is perfectly reasonable. For faster release cycles, we may keep platform resources in Terraform and give the application pipeline a clearly defined deployment responsibility.

We avoid broad ignore_changes rules as a shortcut. They can suppress useful drift detection. Any ignored field should have a documented owner and a reason for being outside Terraform’s control.

Plan for drift, recovery, and cost

Console changes happen, particularly during incidents. We do not want emergency access to become an undocumented alternative deployment process.

After an urgent manual change, we reconcile configuration and infrastructure deliberately. A scheduled plan can help reveal drift, but we avoid automatically applying every detected difference: the manual change may be something we need to preserve.

We also distinguish infrastructure recovery from data recovery. Recreating a database resource does not restore its contents. We need backups, retention decisions, and tested restore procedures alongside Terraform.

For critical resources, we consider lifecycle safeguards and service-specific deletion protection where available. These are guardrails, not substitutes for backups or access control.

Finally, we review cost-bearing defaults: idle databases, NAT gateways, load balancers, storage retention, and data transfer. We configure budgets and alerts, while remembering that budget alerts do not impose a hard spending cap.

Our short readiness checklist

Before calling an environment repeatable, we check:

• Client ownership, billing, and operational responsibilities are documented.
• Each environment has an explicit project, state location, and deployment identity.
• Terraform and provider versions are controlled and lock files are committed.
• Remote state is access-restricted, versioned, and excluded from Git.
• CI validates changes and production applies require approval.
• Runtime identities have only the access they need.
• Backups, recovery steps, cost alerts, and handover instructions exist.

Make handover part of the implementation

Our final deliverable is not just a directory of .tf files. We include a concise runbook explaining prerequisites, authentication, initialization, planning, applying, and recovery. We document configuration ownership and any remaining manual dependencies.

We want another engineer to understand the system without needing a call with the person who created it. For a remote studio, that is good collaboration; for a client, it is practical independence.

If we can help build a repeatable cloud foundation for your next product, start a project. To explore how we connect infrastructure with software, AI, hardware, and design, read our related services pages.