If you haven’t read our first article in the IaaS series, Choosing the Right IaaS Templates, we suggest you start from the beginning.
AWS CloudFormation – Chicken was before the egg
If you regard an IaaS template as your egg to your proverbial infrastructure chicken, then AWS got the process backwards. They started with the infrastructure and later realized it was impossible to manage without templates! I think one of the reasons why CloudFormation is so difficult to use is because they started too late on thinking about what their template engine should be.
With Terraform, the core concepts are clear from day one: Infrastructure is code. Infrastructure as code shouldn’t be limited by an underlying orchestration engine or toolset like CloudFormation does – it should be able to operate in any language and any platform where Terraform has been ported (which is almost all of them).
Frustrations Abound
Clearly, CloudFormation got off to a bad start and drove all of us crazy. It still today is an unwieldy behemoth crashing through our DevOps delicate china shop minds. Actually, it has gotten better over time, but it’s still not perfect. The thing that makes it so tough to use is that we have to think about too many things upfront when we’ve already decided we want to use CloudFormation for our application. There’s a steep learning curve for designing then writing the templates. One piece of guidance here is to take advantage of the AWS Well-Architected website as it is filled with important considerations for your design. Once you have your design worked out, then you have to figure out how to model it in CloudFormation’s structure/syntax.
What the hell is this syntax?
Who what now? XML? YAML? This is like, a higher level computer language era right? Then the “language” of Ref + Sub = . So clunky. However, you can take advantage of a newer library from AWS which uses object oriented programming (a higher level language) to build your templates: AWS CDK.
Developer tooling? Never heard of it!
The first thing you should know about AWS CloudFormation is that it has no way to check what IAM permissions I have to spin up my stack. The second thing you should know is that if they aren’t specified in your template, they will be ignored by CloudFormation. This can result in some really infuriating things happening:
- You’ll need to create an IAM role with all the necessary permissions before running your template.
- Or worse, you’ll need to re-run the same process over again! That’s right – when spinning up a full serverless application using a given template, there are often multiple steps involved (e.g., create an S3 bucket vs provisioning EC2 instances). If something goes wrong along the way, this could mean having to re-run everything from scratch (after waiting for it to hopefully get deleted) – which could be time consuming and difficult if not impossible depending on what’s happening behind the scenes! Just imagine your frustration when CloudFormation can’t delete the resources it setup – manually cleaning things up is not fun.
- Finally, infrastructure drift! Since CloudFormation only sets up your infrastructure but then someone changes something through the AWS console or CLI, CloudFormation doesn’t know about it until you check! Keep in mind that you need to have a process for managing your infrastructure so that administrators/developers codify their changes using CloudFormation.
So what’s good about it?
Once you jump through the flaming hoops of syntax, validation, stack deployment failures and IAM permissions, you can build your infrastructure in a repeatable way! Huzzah! DevOps Nirvana!
You might wonder what your AWS costs might be
It’s a fair question, but also an important one to ask before deploying a CloudFormation template. AWS supports this feature – checkout the CLI estimate-template-cost. You should definitely use this!
Actually does what it says
If you’re running on an AWS infrastructure, then you can spin up new environments for your teams quickly!
Keeping track of your IaaS changes is mind numbing
If you want to version your infrastructure, use your software development practices! You can implement your gitflow to work alongside the code/apps/systems being deployed by your team.
Need extra customization or trigger other workflows?
If you need to do something extra during your deployments, AWS CloudFormation has the concept of Lifecycle Hooks that can be used for this purpose:
- update – which runs after an update or creation and before any rollback occurs. It lets you perform additional tasks that need to be completed during an update.
- create – which runs before a stack create request is sent to CloudFormation, allowing you to prepare data required by the stack such as creating an S3 bucket or DynamoDB table, etc., but not everything will work here (i.e., some resources with `NetworkInterface`, `LoadBalancer`, etc.).
- delete – which runs after the delete request is sent but before it completes so you can perform additional maintenance tasks like deleting files and tables or closing connections/processes/etc.
Lock out the bad guys before spinning up your product
Another huge benefit of using CloudFormation templates is that you get to lock out the bad guys before you spin up your product. No one can hack into a box that doesn’t exist yet, so why not check for security vulnerabilities before it becomes a full-fledged server?
In practice, there are two different ways to do this:
- Automated scanning tools like Snyk or Snyk’s new CLI version (which we’ll call “snyk-cli”) will scan all your dependencies and report on any known vulnerabilities that need fixing. The tooling here is really good! We recommend using them or use AWS’ version called AWS CloudFormation Guard.
- You can roll up your sleeves and run manual vulnerability checks on every part of your codebase with something like Brakeman or WebHawk (or many other tools). This takes a lot more time than automated tools but gives you a better understanding of what’s going on in there. It’s also good practice if nothing else!
Oh yeah, it’s Object Oriented Programming!
AWS CloudFormation allows you to have a single template that is the root controller for your infrastructure and then manage dependencies between other stacks. This architecture enables you to break up your infrastructure into smaller more manageable chunks. For example, if you had an application stack where you wanted to deploy an RDS database, a load balancer and some EC2 instances. You can create these as separate templates but build in references so that when the Application stack is deployed the RDS database will be automatically created at the same time.
Once again this feature really makes it easy to deploy multiple services using a single template file which can then be used as part of any future projects that require similar services. Or use the AWS CDK to build out CloudFormation architecture.
Well architected
CloudFormation is featured heavily in the AWS Well-architected best practices. You should be paying attention to what AWS says!
CloudFormation is useful even to AWS
There’s a good reason why CloudFormation is used by so many companies, including Amazon. In fact, the AWS architecture team even uses CloudFormation to manage their own infrastructure! You can read about how they use it in an article called “Automating Your Way to the Future” on their blog.
Conclusion
CloudFormation is AWS’s engine, you might as well get used to it. It’s hard to get going but after you swim through the fiery depths, you might just find yourself a very handy tool. If you have any comments or questions about this article, please feel free to reach out!