& Continuous Learning

Prerequisites

This article builds on the prior article: Dockertize.

Introduction

In the articles leading up to this one, we've created a docker image of a minimal tested NodeJS microservice. Our next step is to deploy the docker image into a container running in the AWS cloud platform. AWS CloudFormation allows us to create and provision AWS infrastructure deployments predictably and repeatedly. Our deployment will include multiple AWS IAM, ECS, ElasticLoadBalancingV2, S3, ECR, CodeBuild and CodePipeline resources. We'll learn about how to leverage each of these AWS resources in upcoming articles in this series. CloudFormation will allow us to setup these resources by simply specifying them in a YAML file and sending that file to AWS.

Cloudformation

AWS has several ways to setup and configure the services it offers. The AWS Management Console, Command Line Interface, Cloudformation as well as several third party solutions such as Terraform and Serverless allow you to create and configure the components within AWS that are needed to run and support your application.

The easiest and most approachable of these is the AWS Management Console. The Console is a web interface that allows you to manage AWS resources such as S3 buckets, EC2 instances and load balancers. It allows you to "point and click" resources like virtual machines in AWS. Below are a couple of screenshots showing what it looks like when you first login:

And what it looks like when you click on the "EC2" link in order to launch a virtual machine:

The Console is useful for quickly prototyping and exploring AWS services. Many demos and presentations use the Console to setup their applications. However this strategy of manually configuring your AWS resources is time consuming, risky and misses all of the great benefits of capturing in source code how you set them up.

In a cloud-based microservice architecture, every task of infrastructure setup is multiplied across the many services deployed. An organization maintaining a monolith could get away with one database and a few application servers. However, each service in a microservice architecture requires a distinct place to store data, a place to run the application, load balancing to get network traffic to the application as well as networking, firewalls, DNS, and monitoring.

Over time, manually managing so many resources can produce Configuration Drift. This is the phenomenon where different application environments (dev, test, prod) become different from each other as changes are not accurately copied between environments.

Manually configuring environments also does not lend itself well to peer reviews or pull requests. It is difficult to cohesively determine a delta of a desired change.

To avoid the above problems, we need a way to automate the deployment of infrastructure. This is practice is commonly referred to as Infrastructure as Code. By codifying the infrastructure, we can check it into source control and delta changes to it over time. It acts as living documentation for how your application environment should be and can provide compliance auditors with excellent visibility into historical and present state configuration.s

The first inclination would be to use the AWS CLI and write a shell script that creates the desired resources. This works well for the initial setup and prototyping. But, as changes need to occur, the script will get more and more complicated as it is filled with if not exists create, otherwise update logic. What we need is a tool that will determine what configuration changes have and have not been applied and issue the appropriate commands. These tools are generally classified as Configuration Management. Puppet, Chef and Salt Stack are popular options when managing your own physical machines. AWS provides its own configuration management tool called Cloudformation that is simple to use and supports almost all AWS services.

By using CloudFormation we can declaratively define what components our infrastructure requires and how they relate to each other. Cloudformation works by creating a configuration file in either JSON or YAML.

As an example, the NoSQL DynamoDB table we define in our node-reference code accompanying this blog series, we can create a table using the syntax below. The syntax is easily found by googling aws resource + cloudformation which leads you to the official CloudFormation docs for DynamoDB.

Resources:
  ProductsTable:
    Type: 'AWS::DynamoDB::Table'
    Properties:
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: 'HASH'
      ProvisionedThroughput:
        ReadCapacityUnits: 1
        WriteCapacityUnits: 1

We highly recommend using the YAML option over JSON for all templates for the following reasons:

• Template functions (ex. Ref and GetAtt) are much easier to read.
• Comments are supported. Sometimes a parameter has to be set to a specific number or magic value and can be useful to explain to those making future changes why.
• Multiline values are easy to specify.
• It is a lot more concise. Templates can become quite lengthy.

Your CloudFormation template describes the desired state of one or more "Resources" as well as parameters to the template. When CloudFormation is invoked, it is provided with values for the parameters specified in the template as well as a "stack name". A CloudFormation Stack is a logical grouping of assets in AWS that are deployed together. We like to create at least one CloudFormation stack per application. This allows our applications to deploy on their own schedule and gives a single place that documents all the pieces of an application's infrastructure.

CloudFormation then looks at the last set of resources it deployed and their properties. For any resources whose properties have changed it updates or recreates those resources as needed in the order needed.

CloudFormation will not look at the configuration of the resources themselves but rather the last deployed configuration. (To gain more fine-grained control, some prefer to use Terraform). If a user logs directly into the console and manipulates configuration then CloudFormation will not know to correct it unless another change needs to be made to that resource. Because of this, we recommend that access to modify resources in higher environments (i.e. everything but the first environment) be extremely limited.

Wrap-up

In this article, we learned how CloudFormation helps us configure via source code our AWS resources. You can see the full CloudFormation Template for our microservice here.

Table of Contents

• Introduction
• Unit Testing
• Koa
• Docker
• Cloudformation (this post)
• Code Pipeline
• Fargate
• Application Load Balancer
• HTTPS/DNS
• Cognito
• Authentication
• DynamoDB
• Put Product
• Validation
• Smoke Testing
• Monitoring
• List Products
• Get Product
• Patch Product
• History Tracking
• Delete
• Change Events
• Conclusion

If you have questions or feedback on this series, contact the authors at nodereference@sourceallies.com.