SHIP

Improvements and additional components

We are going to create the project using an AWS infra using Infrastructure as code (IaC) Cloudflormation and AWS CLI: ✅

YAML deployment file

Parameters:
  InstanceType:
    Description: micro ec2 instance with health checks
    Type: String
    Default: t2.micro

  KeyName:
    Description: stefano's keypair
    Type: AWS::EC2::KeyPair::KeyName
    Default: stefano-us-east-2

  ImageId:
    Description: aws Image ID for this instance
    Type: AWS::EC2::Image::Id
    Default: ami-024e6efaf93d85776

Resources:  
  MyLaunchConfiguration:
    Type: "AWS::AutoScaling::LaunchConfiguration"
    Properties:
      InstanceType: !Ref InstanceType
      ImageId: !Ref ImageId
      KeyName: !Ref KeyName
      SecurityGroups:
        - "sg-a7f988c5"
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash
          sudo apt-get update -y
          sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
          curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
          sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
          sudo apt-get update
          sudo apt-get install -y docker-ce
          sudo systemctl start docker
          sudo systemctl enable docker
          sudo docker pull cr3w/simple-website:latest
          sudo docker run -d -p 80:80 --name simple-website cr3w/simple-website:latest
          echo '* * * * * root docker pull cr3w/simple-website:latest && docker stop simple-website && docker rm simple-website && docker run -d -p 80:80 --name simple-website cr3w/simple-website:latest' | sudo tee /etc/cron.d/simple-website-update
          wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh &
          wait
          sh /tmp/netdata-kickstart.sh --nightly-channel --claim-token jkrulZscAyB25zUs1dg4HIuI3LOu-5LXqgQRY7tZku9wIwpxdFHjS-iJKQ7GW-WLlBkHtoOYNLtKUvkPqYag6w7ckRPhLz5oQtRHaWn1fKub5zC-08zt_7Tuc5R2Pfriy7grJ4o --claim-rooms 3fa28e8f-9668-4516-a068-989249586164 --claim-url https://app.netdata.cloud
          docker run cloudflare/cloudflared:latest tunnel --no-autoupdate run --token eyJhIjoiNmI2YWQzZDVhOWM2NWY3Y2E5MTViYzZjZTMyZTk3YmQiLCJ0IjoiMDFiNjY3ZWEtYWQzYS00MDNhLWJhYTItZDU1MWY5ZWRhNDM1IiwicyI6IlpXSTFNelZsWXpjdFl6YzNaUzAwTVRZeUxUa3daV1F0T0dJMU1EQmlZelV5TkRCbCJ9
          sudo docker run -d -p 3000:3000 grafana/grafana
        # 1. installing docker, and run a Grafana docker image on 3000 port
        # 2. Creating and running a private tunnel from AWS <--> Cloudflare
        # 3. deploys the app into Cloudflare edge where we 
        # can use other functions if needed ( workers serverless code, zt sec etc)

        # Watchtower
        #  sudo docker run -d \
        #    --name watchtower \
        #    -v /var/run/docker.sock:/var/run/docker.sock \
        #    containrrr/watchtower simple-website \
        #    --interval 300

        # echo '* * * * * root /usr/local/bin/aws ecr get-login-password --region us-east-2 | /usr/bin/docker login --username AWS --password-stdin 693505164922.dkr.ecr.us-east-2.amazonaws.com' | sudo tee /etc/cron.d/ecr-login
        # this is a cript to periodically login to ECR and update Docker's config.json file
        # !/bin/bash
        # TOKEN=\$(aws ecr get-login-password --region us-east-2)
        # echo "{ \"auths\": { \"693505164922.dkr.ecr.us-east-2.amazonaws.com\": { \"auth\": \"\$(echo AWS:\$TOKEN | base64)\" } } }" > /home/ubuntu/.docker/config.json              
        # Restart Docker service to ensure it picks up the new config
        # sudo service docker restart
        # EOF
        # execute the script
        # chmod +x /home/ubuntu/ecr-login.sh
        # /home/ubuntu/ecr-login.sh
        # use cron to run this every 5 hours
        # echo '0 */5 * * * ubuntu /home/ubuntu/ecr-login.sh' | sudo tee /etc/cron.d/ecr-login

        # additinaly Jenkins running on  ssh [email protected] --> http://161.35.223.47:8080/
        # Workflow and Actions on GitHUb: https://github.com/stefanogram/docker-aws/actions


  MyAutoScalingGroup:
    Type: "AWS::AutoScaling::AutoScalingGroup"
    Properties:
      AvailabilityZones:
        - "us-east-2a"
        - "us-east-2b"
      MinSize: "2"
      MaxSize: "2"
      DesiredCapacity: "2"
      HealthCheckType: EC2 
      LaunchConfigurationName: !Ref MyLaunchConfiguration
      TargetGroupARNs: 
        - !Ref MyTargetGroup
      
  MyLoadBalancer:
    Type: "AWS::ElasticLoadBalancingV2::LoadBalancer"
    Properties: 
      Subnets:
        - "subnet-feced596"
        - "subnet-c7dc8dbd"
        - "subnet-52d9611e"
      SecurityGroups: 
        - "sg-a7f988c5"

  MyTargetGroup:
    Type: "AWS::ElasticLoadBalancingV2::TargetGroup"
    Properties:
      HealthCheckIntervalSeconds: 30
      HealthCheckPath: "/"
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 5
      HealthyThresholdCount: 5
      UnhealthyThresholdCount: 2
      Matcher:
        HttpCode: '200'
      Port: 80
      Protocol: HTTP
      VpcId: "vpc-73cb3818"
      TargetType: "instance"

  MyListener:
    Type: "AWS::ElasticLoadBalancingV2::Listener"
    Properties: 
      LoadBalancerArn: !Ref MyLoadBalancer
      Protocol: HTTP
      Port: 80
      DefaultActions: 
        - Type: forward
          TargetGroupArn: !Ref MyTargetGroup




# --------------------------------------------------------------------

# to do --> https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudwatch-alarm.html
# - AWS::CloudWatch::Alarm
# MyCloudWatchAlarm:
#    Type: 'AWS::CloudWatch::Alarm'
#    Properties:
#      AlarmName: 'CloudWatch LB-1'
#      AlarmDescription: 'This is a 'CloudWatch Alarm if CPU utilization exceeds 80%'
#      Namespace: 'AWS/EC2'
#      MetricName: 'CPUUtilization'
#      Statistic: 'Average'
#      Period: '300'   # that will be 5 minutes
#      EvaluationPeriods: '1'
#      ComparisonOperator: 'GreaterThanThreshold'
#      Threshold: '80'
#      AlarmActions:
#        - !Ref MyAutoScalingGroup  # Auto Scaling group ARN -> https://awscli.amazonaws.com/v2/documentation/api/latest/reference/autoscaling/describe-auto-scaling-groups.html
#      Dimensions:
#        - Name: AutoScalingGroupName
#          Value: !Ref MyAutoScalingGroup  # The Auto Scaling group name

Run the script local

To run this simple script local in our machine we will need to run:

aws cloudformation create-stack --stack-name stefano --template-body file://cloudformation.yml

A successful run should respond something like this:

{
    "StackId": "arn:aws:cloudformation:us-east-2:693505164922:stack/stefano/341b0e20-2c9d-11ee-a428-064ee1e828d3"
}

Get the status of the stack

To get the current status and logs of the stack we can use a simple script like the following:

while true; do clear; aws cloudformation describe-stack-events --stack-name stefano; sleep 5; done

Breakdown of the YAML

Breakdown of the YAML

• Parameters: The Parameters section defines the variables that will be used in the rest of the YAML. In this case, the variables are InstanceType, KeyName, and ImageId.

• Resources: The Resources section defines the AWS resources that will be created. In this case, the resources are a launch configuration, an Auto Scaling group, a load balancer, a target group, and a listener. We are using also in the script in our user data, watchtower to watch for changes in ECR, installing Netdata to monitor and send Pager Duty Alerts on server or network failures, we run specific crons, and installing a tunnel to route private and public traffic through Cloudflare.

• LaunchConfiguration: The LaunchConfiguration resource defines the configuration of the EC2 instances that will be created. In this case, the instances will be of type t2.micro, have the Amazon Linux 2 AMI, and be assigned the stefano-us-east-2 keypair.

• AutoScalingGroup: The AutoScalingGroup resource defines the Auto Scaling group that will be created. In this case, the Auto Scaling group will have a minimum size of 2, a maximum size of 4, and a desired capacity of 2. The Auto Scaling group will use the MyLaunchConfiguration resource to create new EC2 instances.

• LoadBalancer: The LoadBalancer resource defines the load balancer that will be created. In this case, the load balancer will use the MyTargetGroup resource to distribute traffic to the EC2 instances in the Auto Scaling group.

• TargetGroup: The TargetGroup resource defines the target group that will be used by the load balancer. In this case, the target group will listen on port 80 and will use the MyLaunchConfiguration resource to determine which EC2 instances to send traffic to.

• Listener: The Listener resource defines the listener that will be used by the load balancer. In this case, the listener will listen on port 80 and will forward traffic to the MyTargetGroup resource.

Testing the apps

1. Download the docker app: https://github.com/stefanogram/docker-aws local

2. Add anything, change the code, and create a Pull Request

3. Wait for an approoval

4. See the change live on one of the IPs/aws hostnames:

--> http://3.15.25.206/

--> http://3.21.231.216/

Health Checks

CloudWatch Alarms

--> CPUUtilization GreaterThanOrEqualToThreshold-CPUUtilization: https://us-east-2.console.aws.amazon.com/cloudwatch/home?region=us-east-2#alarmsV2:alarm/awsec2-i-0a71f3465fd1a1d96-GreaterThanOrEqualToThreshold-CPUUtilization

Additionally on a separate server we run a simple python check script that periodically run a health check to the 2 AWS EC2 URL endpoints see Debugging section for details:

Debugging

ssh

ssh -i "stefano-us-east-2.pem" [email protected]