Cloudformation template with EC2 using docker compose

[Raziyah00]

I'm relatively new to Cloudformation, and had a few questions about my template and best practices. I'm facing a few hurdles, and there is a lot of information out there, it's a bit overwhelming, so any feedback would be highly appreciated. I'm not looking for detailed code etc. just some good insights on how I can improve my steps.

I'm trying to set up a basic Node/Express API:

• On push to Git repo
  • Build Docker image and push to private AWS ECR repo
  • After successful push, deploy Cloudformation template that provisions
    • An EC2 + security group with Elastic IP assigned
    • Run docker compose in Userdata of EC2 to get app up and running

This is my UserData (I do need some specific help here!)

UserData: !Base64 |
        #!/bin/bash -ex
        yum update -y
        yum install docker -y
        service docker start
        usermod -a -G docker ec2-user
        echo "Start Docker service"
        apt-get update
        apt-get install docker-compose-plugin
        apt install amazon-ecr-credential-helper
        echo "APT-GET update complete"
        echo "{ \"credHelpers\": { \".dkr.ecr..amazonaws.com\": \"ecr-login\" } }" > ~/.docker/config.json
        systemctl restart docker
        echo "
        version: "3.9"
        services:
          my-app:
            image: .dkr.ecr..amazonaws.com/my-repo
            environment:
              STAGE: staging
              VIRTUAL_HOST: my-customdomain.com
              VIRTUAL_PORT: 3000
            ports:
              - "3000:3000"
            restart: always
            networks:
              - my-network
      https-portal:
        image: steveltn/https-portal:1
        ports:
          - '80:80'
          - '443:443'
        links:
          - my-app
        environment:
          STAGE: production
        volumes:
          - /var/run/docker.sock:/var/run/docker.sock:ro
        networks:
          - my-network

    volumes:
      https-portal-data:
    networks:
      my-network:
        driver: bridge
    " > docker-compose.yaml
    docker compose up -d

Status: Cloudformation template deploys successfully, all resources set up. But the Userdata doesn't run, so my EC2 never sets up my app.

Issues / Questions:

• The Userdata never ran, I can't find see any of the above echo statements in the logs /var/log/cloud-init.log. When I SSH into the instance I can't find any of these files. How do I debug this better?
• Is there a better way to get the docker-compose data in there? writing the whole file in the UserData script seems inefficient. Is there a better way to do this?
• On code update, Cloudformation stack is updated, this does not run Userdata(?) (I know it only runs when an instance is first created, but I would like some confirmation that Cloudformation update does not trigger this.
• What is the best practice here if I want to re-run docker compose in my Ec2 after every Cloudformation deploy? If it does trigger Userdata, what could be wrong here?
• Is this an ideal flow? Are there any improvements I can make here, considering I'm not an expert, but willing to spend some time learning where required.

I appreciate anyone taking the time to answer these questions. Thanks!

[Amritpal]

So I found some answers over time, posting them here in case it helps anyone.

• The Userdata never ran, I can't find see any of the above echo statements in the logs /var/log/cloud-init.log. When I SSH into the instance I can't find any of these files. How do I debug this better?

  ANS: There was an error in my Userdata, I SSHed into the instance and ran the script manually to debug and fix all issues.

• On code update, Cloudformation stack is updated, this does not run Userdata(?) (I know it only runs when an instance is first created, but I would like some confirmation that Cloudformation update does not trigger this

  ANS: Cloudformation terminates the old instance and spins up a new one on each deploy, this causes the Userdata to be run each time.

• Is this an ideal flow? Are there any improvements I can make here, considering I'm not an expert, but willing to spend some time learning where required.

  ANS: It was not. The primary reason I was doing this was to fire async requests from one lambda to another, without waiting for the results. This is not possible, as the lambda will not wait for any requests that you do not await meaning you still have to wait for those operations. I tried asynchronous invocation but it still did not work (For those interested I posted a question about it https://stackoverflow.com/questions/72493614/serverless-express-lambda-async-invoke-not-working ). So I switched to using SNS to publish messages from one lambda to trigger another, achieving my requirement for fire and forget requests.