Infrastructure as Code (IaC)
Terraform
- Previously, I used Terraform for IaC. I’m not a fan! You can read about my experience here
My strenuously-crafted, resuable, immutable, versioned infrastructure modules and artifacts became a big sphagetti monolith covered in a sauce of a thousand inconveniences, like security groups and IAM roles which broke any attempted updates by refusing deletion
CloudFormation setup
- For this project, I used AWS CloudFormation - the basic setup was four modules:
- VPC:
- VPC, Subnets, Route Tables, Internet Gateway
- Backend:
- Load Balancer, Listener Rules, Target Groups, ECS Cluster and Service, IAM roles, and Security Groups
- Frontend:
- S3 Bucket, CloudFront distribution
- Lambda Functions:
- Lambda Function, Execution Role
- VPC:
- Deployed to separate AWS accounts for:
- Staging
- Production
- One other very useful approach was using ClickOps on occasion and writing down what was done in the README
Conclusions
- It was convenient!
- The CloudFormation YAML files were generally easier to read and write than the Terraform HCL files
- Plus - the modules don’t depend on each other so much, which made them easy to reason about and update
- And when they do depend on one another, the remote state is easy to find and always correct because it’s just in the AWS console and reflects the actual deployed infra
Simpler beats complexer
- The mostly-independent, AWS-only CloudFormation stacks may seem a bit too simple and with too much vendor lock-in
- A comprehensive any-cloud Terraform solution may seem really cool and fancy
- But to that I invoke…
- Two of Werner Vogel’s Internet Technology Architecture principles:
- “Use late bindings”
- “Decompose into small, loosely coupled, stateless building blocks”
- Two of Werner Vogel’s Internet Technology Architecture principles:
- What about non-AWS resources?
- Well, one answer is to just use AWS. But mabye GCP and Azure have a “CloudFormation” product too? I’d try that first (and Pulumi and any other vendor) before going back to Terraform
- Full Series: CHRT - Autopilot for Analytics (6 posts)