Multi-Cloud Terraform CI/CD Governance Pipeline

This project showcases a governance-first Terraform CI/CD pipeline built with Azure DevOps, designed to validate and plan infrastructure changes across Microsoft Azure and Amazon Web Services (AWS) without automatically applying them.

The pipeline prioritizes change visibility, safety, and predictable outcomes, following enterprise practices where infrastructure deployments are reviewed before execution.

Architecture Overview

The pipeline follows a validation-only workflow. Terraform plans are generated for each cloud provider independently while sharing a common CI/CD governance model.

┌────────────────────────┐
│      Developer PR      │
└───────────┬────────────┘
            │
            ▼
┌────────────────────────┐
│   Azure DevOps CI/CD   │
│  (Validation Pipeline) │
└───────────┬────────────┘
            │
            ├────────────► Terraform Validate
            │
            ├────────────► Terraform Plan (Azure)
            │                │
            │                ▼
            │        Azure Blob Storage
            │           (Remote State)
            │
            └────────────► Terraform Plan (AWS)
                             │
                             ▼
                    Amazon S3 + DynamoDB
                       (State & Locking)
  

Terraform State Management

Remote state is used for both platforms to support predictable execution, collaboration, and protection against concurrent runs.

Azure Backend

terraform {
  backend "azurerm" {
    resource_group_name  = "rg-terraform-state"
    storage_account_name = "tfstatemulticloud"
    container_name       = "tfstate"
    key                  = "azure/infra-cicd.tfstate"
  }
}
  

AWS Backend

terraform {
  backend "s3" {
    bucket         = "tfstate-multi-cloud-infra-cicd"
    key            = "aws/infra-cicd.tfstate"
    region         = "us-west-2"
    dynamodb_table = "terraform-state-locks"
    encrypt        = true
  }
}
  

Provider Configuration

Provider configuration is intentionally minimal and environment-specific. Authentication and credentials are handled by Azure DevOps service connections, not hardcoded values.

Azure Provider

provider "azurerm" {
  features {}
}
  

AWS Provider

provider "aws" {
  region = var.region
}
  

CI/CD Pipeline Behavior

• Terraform Validate – Ensures syntax and provider correctness
• Terraform Plan (Azure) – Generates a reviewable Azure plan
• Terraform Plan (AWS) – Generates a reviewable AWS plan
• No terraform apply stage is included by design

Terraform plans are published as pipeline artifacts, enabling human review before any infrastructure changes occur.

Issues Encountered & Resolutions

AWS Credential Injection Failures

Terraform initially failed with NoCredentialProviders errors when running in Azure DevOps. Although AWS credentials were available locally, they were not reliably injected into the CI pipeline when using generic Bash tasks and variable groups.

Resolution: The pipeline was updated to use an AWS service connection and the AWSShellScript task from the AWS Toolkit for Azure DevOps. This ensured credentials were injected before terraform init, allowing the S3 backend and DynamoDB state locking to initialize correctly.

Terraform Backend Initialization on Ephemeral Agents

The pipeline initially failed during backend configuration because Azure DevOps uses ephemeral build agents with no cached Terraform state or providers.

Resolution: The AWS plan stage explicitly runs terraform init -reconfigure on every pipeline execution. This forces Terraform to reinitialize the backend and download providers consistently.

YAML Validation and Task Schema Errors

Several pipeline executions failed before runtime due to YAML structure and indentation issues, which prevented Azure DevOps from determining whether the pipeline should execute.

Resolution: The pipeline YAML was restructured to align strictly with Azure DevOps schema rules, ensuring task definitions, variables, and stages were correctly scoped and validated using Azure DevOps’ built-in YAML validator.

Why This Approach Matters

In enterprise and regulated environments, uncontrolled infrastructure change is often a greater risk than lack of automation. This project demonstrates how CI/CD pipelines can function as guardrails, enforcing validation, review, and accountability across multiple cloud providers.

Technologies Used

• Terraform
• Azure DevOps Pipelines
• Azure CLI
• AWS Toolkit for Azure DevOps
• Azure Blob Storage
• Amazon S3 & DynamoDB
• Infrastructure as Code (IaC)