There are a few changes that I'd make. First, I'd get rid of the QA sign-off on dev before cutting a release branch. I'd look at methods to instill a culture of developer-led testing (especially if that means developing automated tests) on your trunk. Of course, this doesn't mean that your testers shouldn't be using the trunk deployed to the development environment - they can be practicing any manual test cases or doing exploratory testing and giving feedback. Second, if you haven't, I'd look at automating the end-to-end testing done in the UAT environment, at least from a regression standpoint. You may want to do some manual testing, especially of an exploratory nature, in UAT, but you want to reduce the burden of manual testing, especially as the system increases in complexity. It may not matter much, but I would recommend fixing defects in the release branch and merging back into the trunk. You could also cherry-pick or rebase or some other method, as well. But I've found that going from release branch to development branch is more intuitive. Once you have a sign off on the release, apply a tag. You can either get the state of the code back into trunk and deploy the tag in the trunk or you can deploy the head of the release branch. Both are the same thing. If you have multiple UAT in progress at once, then you may need multiple environments. However, you're also introducing complexity around a defect found in UAT2 that also impacts UAT1 and managing getting the fixes synchronized. I'd want to understand what makes UAT take so long and what can be done to get an accepted system into production faster to reduce parallel UATs.

Sorta, but the additional process it does is compilation of the source code (or linking to a jenkins agent that compiles the code). Then it can deploy the finished build to wherever you tell it. This could be just outputting the build to a network share, or it could be sending it to a virtual machine or docker container to run and test for bugs

The first thing to do is to define what is meant by "Continuous Integration", "Continuous Delivery", and "Continuous Deployment". They've come to mean different things to different people. Continuous Integration comes from Extreme Programming, developed by Kent Beck and others. In Extreme Programming Explained: Embrace Change, Beck says this about Continuous Integration: Integrate and build a complete product. If the goal is to burn a CD, burn a CD. If the goal is to deploy a web site, deploy a web site, even if it is to a test environment. Continuous integration should be complete enough that the first deployment of the system is no big deal. From an Extreme Programming perspective, Continuous Integration would result in a deliverable, such as a library or package or container in a registry. If the goal is to make the package, that would be sufficient. However, if the team handles building the system, true Continuous Integration would go further and deploy that package or container to an environment. The team would need to define what its goal and deliverable would be - is it a user-facing software system or a package? Having a well-controlled staging or test environment is almost certainly required when performing Extreme Programming's form of Continuous Integration or what is often referred to as Continuous Delivery today. These practices require that you have confidence in the ability to deploy software to the production environment on-demand. Without deploying it somewhere, preferably to an environment that is sufficiently like the production environment, how do you intend to have the necessary confidence? This only applies to teams building systems, though - teams building packages may be confident by creating a package somewhere and not publishing it. As far as branching, you don't need a branch-per-environment. You can continue to use trunk-based development ( https://trunkbaseddevelopment.com/committing-straight-to-the-trunk/ or https://trunkbaseddevelopment.com/short-lived-feature-branches/ ) with https://trunkbaseddevelopment.com/release-from-trunk/ or https://trunkbaseddevelopment.com/branch-for-release/ . If you are practicing XP's definition of CI or Continuous Delivery, each commit to trunk would result in a build created and deployed somewhere. When you decide to go to production, a person would make the decision to promote the artifacts associated with a specific build to production. If you are branching for release, this would be associated with creating a release branch from trunk. Otherwise, it would be promoting a particular commit hash or tag.

Does this procedure make sense as a solution to my use case? vscode devcontainers are one of the greatest inventions in the DevOps space and I can't believe more people are not talking about them. I think having the local dev environment and the pipeline in sync makes a lot of sense. Though I think the procedure you outlined above could be simplified. I can't imagine being the first to realize this use case. Is there an existing mechanism (as part of the docker utilities, as part of the Azure DevOps Pipelines framework or from something completely different) that fulfills my needs? I'm currently doing this in some of my https://github.com/DontShaveTheYak/terraform-module-template/blob/main/.github/workflows/test.yml . It is also supported on https://github.com/devcontainers/ci/blob/main/docs/azure-devops-task.md .

The problem is that an official Docker image of phpstan uses ENTRYPOINT ["phpstan"] and the Gitlab runner call the image with "sh -c" command. You can override the default entry point ( https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#overriding-the-entrypoint-of-an-image phpstan: stage: check image: name: ghcr.io/phpstan/phpstan entrypoint: [""] script: - phpstan analyse I recommend to use a Docker image for phpstan with concrete version (don't use latest tag).

This question risks being flagged as "too broad", and invites opinion-based answers. With that caveat out of the way I'll try to answer in an unbiased way. In these cases, it's best to go back to first principles and architect the application from scratch using the relevant AWS services. If you're deploying an application that is really just a few services that can be composed, you're probably best off using Fargate. In any case, I would not suggest using EC2 instances In terms of how to go about doing that: Create the VPC if you need it define security groups that will define what traffic is allowed set up the ECR repositories for the images in the application stack convert the services in the docker compose file to ECS task definitions decide how you will manage persistent data? If part of the application is redis, you might consider using an ElasticCache component in the stack. decide how you want to expose the services (ENI? Load balancer?) There is also the issue of IAM policies that would need to be attached to various entities, in order to read / write to ECR, etc. I would write this all as one or two Terraform modules. One for the network configuration (VPC, subnets) and one for the application itself (ECS cluster, Capacity providers, load balancers, ECR repositories, etc). This is a purely personal choice though, and you could just as well create a CloudFormation stack. Continuous deployment could indeed be done with Github Actions, if that's where you are storing the code which describes the app.

Continuous Testing is a https://en.wikipedia.org/wiki/Continuous_integration of Continuous Integration. CI's main purpose is to integrate your development efforts into a production-ready state as often as possible. The production-readiness is granted via continuously testing the recently integrated code. Continuous Integration does not work properly without Continuous Testing, but you could continuously test without making your development code production ready all the time. In my experience the CT term is not really in use anymore.

You can use rules in gitlab ci to control when you want to stage to be executed: https://docs.gitlab.com/ee/ci/yaml/index.html#rules e.g. set rules to only publish package when branch name is master. You can even use regex for filtering branch name. Hope it helps.

This is guaranteed to always fail: run: | [ "$GITHUB_REF_NAME" = Cypress-reconfigure ] && echo $STAGING_ENV_FILE | base64 --decode > .env [ "$GITHUB_REF_NAME" = staging ] && echo $PRODUCTION_ENV_FILE | base64 --decode > .env Why? Depending on the value of $GITHUB_REF_NAME only one of the lines will run. But the other line will return a non-zero return code: $ [ "a" = "b" ] && true $ echo $? 1 And GHA is configured to abort on the first command that doesn't return a zero exit code (that's the successful exit code). To fix, you should use an if-then, rather than the shell one-liner: run: | if [ "$GITHUB_REF_NAME" = Cypress-reconfigure ]; then echo $STAGING_ENV_FILE | base64 --decode > .env; fi if [ "$GITHUB_REF_NAME" = staging ]; then echo $PRODUCTION_ENV_FILE | base64 --decode > .env; fi Note that I didn't include a different workaround in the shell command of ignoring errors with a || true: [ "$GITHUB_REF_NAME" = Cypress-reconfigure ] && echo $STAGING_ENV_FILE | base64 --decode > .env || true because it results in false positives if the command (e.g. base64 --decode) fails.

The pipeline is configured correctly, you were just missing one piece of the puzzle called https://docs.github.com/en/actions/using-containerized-services/about-service-containers . Basically, the service containers also known as sidecars will run alongside your "main" pipeline logic and support your commands. In your case, you will run the npm run dev in a service container once the build job starts and when the npx cypress run runs it will have a dev server to connect to. Follow the example GitHub added for https://docs.github.com/en/actions/using-containerized-services/creating-redis-service-containers

In order to get single /etc/localtime symlink or file from image, you have to download the whole layer anyway. To deterimine which layer contains latest /etc/localtime, you have to download all layers of image. When you downloaded all layers, it's really simpler to just run container with an entrypoint changed to date +%z: docker run --rm --entrypoint /bin/date +%z +0000 It will not consume many resources.

TL;DR: See bold sentence below. I know the separation of concerns (SoC) principle mostly from a "software code organisation" point of view. I never explicitly encountered it for build job definitions up to now (which does not mean it is bad to follow it in this context too). From my point of view: Principles and good (or even best) practices exist for a reason. Following them can help you guide your project to success, because they seem to appear often in successful projects[citation needed]. This does not mean that every successful project follows all best practices. On the other hand, following them does not automatically make your project successful.[citation needed, too] Please note that I don't know building Docker images with GitLab well enough to give you advice in this direction. Maybe GitLab provides some special mechanism directly for this use case? In your case: The main goals of SoC are to manage complexity, increase modularity and facilitate maintenance. I summarise the points you give in your question as increase pipeline efficiency. So the questions are: Does "test during build" increase the performance so much that it pays the additional maintenance costs? How complex are the building and testing logics for themselves? How easy is it to split them apart again if you put them together now? To put in the two additional principles "Premature Optimization" and "You Aren’t Gonna Need It": Do you really need this efficiency now? On the other hand, do you really need the separation now? Maybe it is possible to do both, separate building and testing each in their own script files but call them both from one job? If you cannot find strong arguments in favour of one or the other now, do as you like and be prepared to refactor later when your needs become clearer. Whatever you do, document your decision and the reasons for it together with the alternative as comments in your pipeline definition. Example: Build and test are done in a single job to avoid pushing/pulling the Docker image to/from registry to increase performance. If the complexity of build/test logic increases too much, split them in separate steps and check if performance is still acceptable. This way you and your colleagues in the future can better understand why it is the way it is currently and whether it makes sense to change it.

Set the LOGURU_COLORIZE variable in your Docker container. run: stage: run image: $CI_REGISTRY_IMAGE:latest variables: LOGURU_COLORIZE: "true" script: - ls -la - python3 colored_logging.py

I'll give some general suggestions and specific example how we are doing it - hope it will help. First of all, your idea to have a separate deployment repository is good and it is considered best practice in most cases. Next, it makes more sense to organize things on artifact level rather than code level - essentially, your deployment repository would have records of artifacts rather than code pointers via say git submodules. Personally, I try to stay away from git submodules whenever possible as they complicate things a lot. To elaborate on the above, each of your existing individual repositories would produce artifacts (docker images) via CI, place this artifacts to some registry, and then you would reference those artifacts in your deployment repository. Now, the basic ways of maintaining artifact changes in deployment repository is manual edits of artifact identifiers, which can be simplified by referencing via tags. I.e., you reference specific artifact as myartifact:2 - and then all version 2 artifacts would automatically make it to release. Say you start with 2.0.0, then add 2.0.1 - it will be picked accordingly. Note, that referencing via tags is fine for testing, but not recommended for production - for production we always recommend specific referencing via sha256 digest of the image which ensures uniqueness and immutability. To simplify the above tasks, people may put all artifact images into helm values files or use Kustomization with all image identifiers in one place. Some pull request logic may also be put on top (which is not always needed in the fully automated way below). Finally, there are fully automated ways. The way we do it is we use automation via Reliza Hub - tool we are working on - to glue everything together. So in the nutshell it tells your helm chart what artifacts to use in the automated way. Here is my toy project that shows various paths around the concepts above: https://github.com/taleodor/mafia-deployment

Continuous Testing is an important part of a fast development process. You can detect bugs much faster and find them more easily since the code changes are smaller. But that's independent from integrating my feature into the product of the main/master branch. I could also test very often on my feature branch and never integrate it. So it is a requirement for Continuous Integration but it's not restricted to only that integration model. The difference between the two definitions of Continuous Integration is not that big. Both state, that you should integrate your features rapidly or multiple times a day. If you cut your features into smaller parts, you can also merge them more often into your main/master branch. You should write your code in a non breaking way, to be able to integrate often. The continuous testing helps a lot to ensure, that the changes don't break anything. The important part of CI is, that you get immediate feedback on your code and that each developer is working with the same code base. So you don't have bigger problems with merging of huge feature branches at the end of the development. The problem with huge feature branches and late integration is the following: You finish your development and test everything on your feature branch. When you merge it into the main/master branch, the build breaks because there are some changes on main/master that are incompatible with yours. So you need to recode some parts, redo the tests and retry the merge. This could be prevented, if everyone would be using the latest codebase and all changes are integrated into that codebase continuously. You can also use trunk based development ( https://trunkbaseddevelopment.com/ ) as an extreme form of CI. All work is done on the main/master branch, that is called trunk in this model. Extensive automated tests ensure, that your commits don't break anything. There is no organisation that has the ownership of the concept Continuous Integration, so everyone could publish his or her own definition of it.

Multi branch pipelines work great when your are using the git flow method: https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow We define one multi branch pipeline per application. There is one pipeline documentation for all branches. Based on the branch name different artifacts are build in the end. Main branch builds release artifacts. This could be replaced with builds on git Tag events. The release artifacts will be deployed by fixed version numbers to the test and production environment. Develop branch builds Snapshot artifacts. Those will be deployed by using latest to the develop environment. Feature branches build also Snapshot artifacts which can be deployed to the dev environment as well. All commit events on all branches run lint, unit test, SAST, dependency scan checks before the artifact is pushed to the artifactory. This guarantees that you don’t have to remediate a lot of findings after merging into the develop or main branch. Using different artifactories helps you reduce storage costs because you can just delete all Snapshot artifacts after 30 days. Feel free to ask for specific details. Cheers!

One could use Gitlab templates in conjunction with an environment variable that sets the python version.

You could use Major Public Cloud Providers (AWS, Azure, GCP) or any other Location (Hetzner Cloud, Linode, digitalOcean etc) [ https://www.packer.io/docs/builders ]that is supported by https://www.packer.io/guides/hcl . With Packer you can build and maintain your system image and customize it. Each time you want to upgrade you can build a new image. By building the new image you can choose the base image version you will build on run your modifications on. Image is saved on the provider. Finally you deploy your new server with the image you created and finally you dispose the old infrastructure, with the help of https://www.terraform.io/ . Since packer read code you treat it like any other code.

if: $SHOULD_COPY_ASSETS https://docs.gitlab.com/ee/ci/jobs/job_control.html#check-if-a-variable-exists , not its value. You want something like if: $SHOULD_COPY_ASSETS != "false"

According to Travis documentation, https://docs.travis-ci.com/user/languages/php/#Default-Test-Script :The default test script is phpunit.If your project uses something other than PHPUnit, you can override our default test command to be anything you want.You can change that. https://docs.travis-ci.com/user/customizing-the-build/#Customizing-the-Build-Step :script: - do something - do something else