Mastering CloudFormation and YAML: Building Efficient Cloud Infrastruc

Have you ever been curious about Infrastructure as Code (IaC) and how it simplifies the process of provisioning and managing resources in the cloud? Look no further! In this article series, we'll delve into CloudFormation, a powerful service by AWS that allows you to define and deploy your cloud infrastructure using code.

What to Expect in This Series

Are you ready to embark on a journey into the world of CloudFormation and YAML? In this comprehensive article series, you'll:

Learn the Basics: We'll kick things off by understanding the foundational concepts of CloudFormation and YAML. What are templates, stacks, parameters, and resources? We'll demystify these terms.
Hands-on Examples: With hands-on examples, you'll discover how to create various AWS resources using CloudFormation and YAML. From EC2 instances to DynamoDB tables, you'll gain practical experience in defining your infrastructure as code.
Advanced Topics: As we progress, we'll tackle more advanced topics like using nested stacks, layered stacks, etc.

Prerequisite

Familiarity with YAML
Basic Understanding of AWS Cloudformation

Infrastructure As Code(IAC)

Before the advent of Infrastructure as Code (IAC), we used to manually create cloud resources such as Lambda Functions, API Gateways, VPCs, and more using the service console within AWS Console. The AWS Management Console is a web-based user interface provided by Amazon Web Services (AWS) for managing and interacting with various AWS services and resources.

AWS Management Console

However, this manual process can become cumbersome and time-consuming, especially when dealing with multiple AWS services that need to be created one by one. To address this challenge, the concept of grouping desired resources and provisioning them as a single unit emerged. This not only streamlined the deployment process but also improved consistency and version control by codifying the infrastructure configuration. This practice, known as Infrastructure as Code (IAC), enables developers to define and manage their cloud resources through code, providing more efficient and automated ways to deploy and manage.

We use YAML(YAML Ain't Markup Language) or JSON(JavaScript Object Notation)code to write configuration files.

Why CloudFormation and YAML?

CloudFormation is an essential tool in the world of DevOps and cloud computing. it is an AWS IAC service that empowers you to create and manage resources seamlessly through code, making your cloud deployment more efficient, repeatable, and less error-prone. In this series, we'll focus on CloudFormation's core concepts and explore how it enhances your cloud engineering skills.

Cloudformation Template is a JSON or YAML file where you define the infrastructure resources you want to provision and configure in the AWS cloud. This template is sent to AWS Cloudformation to help create the requested resources according to the configuration defined in the template

When those templates are sent to Cloudformation, a STACK is created, a grouping for all the resources that have been provisioned, deployed, or created. A stack is a collection of AWS resources that you can manage as a single unit.

YAML (YAML Ain't Markup Language) is the chosen language for writing CloudFormation templates. YAML's simplicity and readability make it an excellent choice for expressing complex cloud configurations in an intuitive manner. Whether you're a seasoned developer or just starting out, understanding how to craft YAML templates will open doors to efficient cloud infrastructure management.

Now we have learned what CloudFormation, Template and Stack mean we will dive into the different elements that make up a cloud formation template

Elements of CloudFormation Template

Resources

This is a part of the template where you define the AWS resources you want to create, by specifying the name of the resource, the type of resource, and the resource properties.

Below is an example of declaring a resource element for an EC2 instance
The identing explains the hierarchy, Note the identing!!
```
 Resources:
   WebServerInstance:
     Type: 'AWS::EC2::Instance'
     Properties:
       InstanceType: t2.micro
       ImageId: <replace with AMI ID ami-xxxxx> eg. ami-12345678
```
- we defined the resource "WebServerInstance"
- This resource name has "Type" and "Properties" Attributes.
- The type is specific to different AWS services, using the Type specified in the YAML file above tells AWS that you want to create an EC2 Instance. you can find the other AWS resource Type in the official docs
- The properties needed to create this EC2 Instance are "InstanceType" and "ImageID"
Format Version

The "Format Version" field is defined at the top of the CloudFormation template and it is used to specify the version of the CloudFormation template format being used.

By specifying the"Format Version" you ensure that your template is processed correctly and consistently by AWS CloudFormation, regardless of potential updates to the CloudFormation template format.
```
 AWSTemplateFormatVersion: '2010-09-09'
```
Description

This is optional but just as the name implies it helps to give a more detailed description of the resource you are creating.
```
 Description: An EC2 instance to be used for a group Project
```
Metadata

This is additional information and configuration that you can attach to the resources defined within the template that can be useful for understanding, managing, and automating your infrastructure. You can include links to documentation, cost allocation custom tags for billing purposes, descriptions, etc within it.

This data isn't used by AWS CloudFormation itself during resource provisioning but can be used by other tools or processes.
```
 Metadata:
   Tags:
      Key: Name
      Value: EC2 instance
```
- I used Metadata to create tags for my resource, this will make it easy to track the created resource billings.
Parameters

In CloudFormation, parameters allow you to declare variables that users can input when creating or updating a stack. They provide a way to make your templates more dynamic and customizable, allowing users to specify different values based on their requirements. You can use parameters to pass sensitive data, such as passwords or API keys, without including them directly in the template. This helps maintain security by keeping sensitive information out of the template and also allows for more customization and reusability of your templates.

For instance, in the following parameter definition:
```
 Parameters:
   InstanceType:
     Description: EC2 instance type
     Type: String
     Default: t2.micro
     AllowedValues: [t2.micro, t2.small, m4.large]
```
- We have defined a parameter named 'InstanceType' which allows users to specify the EC2 instance type.
- The attributes within this parameter definition, such as "Description", "Type", "Default", and "AllowedValues," provide additional information and constraints for the parameter.
- "Description": Describes the purpose of the parameter.
- "Type": Specifies the data type of the parameter (in this case, it's a string).
- "Default": Provides a default value for the parameter if the user doesn't provide one.
- "AllowedValues": Defines a list of allowed values that the user can choose from when entering the parameter value.
Intrinsic Functions

Intrinsic functions are built-in functions that you can use to perform various operations and transformations on the template. This helps us reduce code duplication, improves code readability, breaks down complex tasks into smaller, manageable pieces of code as well as provide dynamic capabilities that enable you to create versatile and adaptable infrastructure templates.

They are several intrinsic functions but I will be discussing the most used ones below

Note: We call different functions in our YAML file by specifying the name of the function with an exclamation mark in front of it. eg,!Ref,!Join
- Ref: The Ref intrinsic function is used in AWS CloudFormation templates to retrieve the logical name or ID of a resource that has been defined within the template. This means it provides a way to reference the logical name you assigned to a specific resource, allowing you to use that resource's properties or attributes.
  
  Example: In the following example, we are referencing the logical name given to the resource we created which is "MyInstance" using !Ref MyInstance. This allows us to use the logical name "MyInstance" to access the properties and attributes of the EC2 instance resource we've defined subsequently.
```
  Resources:
    MyInstance:
      Type: AWS::EC2::Instance
      Properties:
        BucketName : my-EC2-Instance
        InstanceType: t2.micro
        KeyName: my-key-pair
        SecurityGroups: [my-security-group]

  Outputs:
    InstanceTypeOutput:
      Value: !Ref MyInstance
```
- Fn::GetAtt
  
  This works hand in hand with the Ref function, this means that before we use the GetAtt function we must call the Ref function to enable the system to understand that you are accessing the property values from a specified resource.
  
  "!GetAtt" helps us to retrieve the value of the property we have defined.
  
  Example:- The example below retrieves the value from the specified property "BucketName" which returns "my-EC2-Instance" as an output.
```
  Resources:
    MyInstance:
      Type: AWS::EC2::Instance
      Properties:
        BucketName : my-EC2-Instance
        InstanceType: t2.micro
        KeyName: my-key-pair
        SecurityGroups: [my-security-group]

  Outputs:
    InstanceTypeOutput:
      Value: !Ref MyInstance
      BucketDescription: !GetAtt MyInstance.BucketName
```
- Fn::Sub
  
  The Fn::Sub function in AWS CloudFormation is indeed similar to template literals in JavaScript programming. Just as template literals allow for string interpolation or substitution within a string in JavaScript, the intrinsic function here is used in CloudFormation templates for variable substitution within a string.
  it uses "${}" with "!Sub" syntax for its operation and an example is used below to tell AWS to fetch the region where the stack is being created.
```
  Resources:
    MyBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: !Sub my-unique-bucket-${AWS::Region}
```
- Fn::Join
  
  This is used for concatenation of multiple strings together, creating a single string and it works by specifying a delimiter. In CloudFormation, we use - as the delimiter.
  
  Example: From the example below we used the "!Join" function to give a value to the "BucketName" property and this value will be returned as "my-unique-bucket"
```
  Resources:
    MyBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: !Join
          - "-"
          - ["my", "unique", "bucket"]
```
Pseudo Parameters

Pseudo parameters are predefined variables that you can use in your AWS CloudFormation templates to access information about the stack and resources during stack creation. These parameters are automatically available in any CloudFormation template and don't need to be explicitly declared or defined.

Some common Pseudo Parameters AWS provides are:
- AWS::AccountId: Returns the AWS account ID of the account in which the stack is being created.
- AWS::Region: Returns the AWS region in which the stack is being created.
- AWS::StackId: Returns the unique ID of the stack.
- AWS::StackName: Returns the name of the stack.
- AWS::NoValue: Used to indicate that a property doesn't have a value during stack creation.
  
  Note: The "!Sub" function is used to call these Pseudo Parameters in our code.
```
  Resources:
    MyBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: !Sub my-unique-bucket-${AWS::AccountId}
```
Outputs

In CloudFormation, the "Outputs" section is where you define values that you want to make available once the stack is created or updated. These outputs can include information about resources created in the stack, computed values, or references to other resources.

Outputs can be used for debugging purposes because they enable us to get access to information about resources within a stack and we output values in our cloud template by specifying them in the "Outputs" Parameter.

This example below outputs the PublicDnsName of the "WebserverInstance" resource created.
```
 Outputs:
   EC2PublicDNS:
     Name: !Ref WebServerInstance
     Description: 'Public DNS of EC2 instance'
     Value: !GetAtt WebServerInstance.PublicDnsName
```
Flipping Formats

One of the convenient aspects of working with Infrastructure as Code (IaC) using CloudFormation is the flexibility to switch between different configuration file formats, whether it's JSON or YAML.

If you're comfortable with one format but need to switch to the other, you don't have to rewrite your entire configuration from scratch. The flipping format codes allow you to smoothly transition from one format to another without compromising the structure or content of your configuration.
You can do that by following these instructions and running the following commands
- install with Pip
```
  pip install cfn-flip
```
- or install with homebrew on MacOS
```
  brew install cfn-flip
```
- Verify the installation by running
```
  cfn-flip --version
```
- then convert from source to target.
  
  Here we are flipping the JSON file to the YAML file. replace the placeholders with the actual name of your file.
```
  cfn-flip <json-file> <yaml-file>
```

Challenge Task

We will be creating a DynamoDB AWS resource using CloudFormation Template and YAML and applying the different elements of the Cloudformation template we have learned.

SOLUTION FOR DYNAMODB

Write out your configuration in a YAML file within your code editor

we specified the template version with a Description.
Created two Parameters "PrimaryKeyElementName" and "PrimaryKeyElementType"
created our Resource which we gave the logical name "ContactEntriesTable"
within our Resources, we have three attributes; "Type", "Properties" and "Metadata"
within our Properties attribute, we defined the necessary configuration for our tables which are; "TableName", "AttributeDefinitions", "KeySchema", and "BillingMode"
the TableName specifies the name of our DynamoDB table
the AttributeDefinitions should only contain the key attributes, here we specified our table's PRIMARY KEY using the "!Ref" function.
The KeySchema is used to specify which one of the AttributeDefinitions will serve as the primary key for our table.
And here we are using the "ID" and it should be hashed for security purposes.
The BillingMode is set to on-demand for cost optimization**.**
Then we have the Metadata where we specified a tag name and value for cost & billing purposes
And finally, we have the Outputs, that will provide the name of our stack after the creation.

We have various options for uploading our YAML file to cloudformation, you can use your local device which has AWS CLI installed, or upload to cloudformation using the ServiceConsole.

For this challenge, we will be using the Service Console.
- Login to your AWS Account and open CloudFormation Console. Then click on Create Stack
- Upload your YAML file, name your Stack, and leave other configurations as it is.
- Then wait for a few minutes for the status to show completed
- Inspect the different Cloudformation tables to be sure there are no errors.
- Check the DynamoDB console for the created table
  
  Congratulations on making it up to this point!! we practiced some of the CloudFormation elements we learned previously and we will dive in more on the elements in the next article on the series.