Yes, there is a supported method of doing this. You can install https://wiki.openstack.org/wiki/Magnum . This will allow you to install any Container Orchestration Engine (COE), including Swarm, Kubernetes, Mesos, and DC/OS. It does this by provisioning an infrastructure cluster of raw OpenStack resources like Nova (the OpenStack compute resource) per-tenant, and installing the COE on the top of it giving you access to the cluster through kubectl. Links https://wiki.openstack.org/wiki/Magnum https://www.youtube.com/watch?v=nxPUd8ZZuD0

Most likely the best path forward is to provision the VMs using Terraform, and the rest with Ansible. OpenStack can be set up a number of ways, but I believe the only hook in the terraform provider that would allow you to execute code on the node would be via cloud-init user data (bash script). What I would do is provision nodes with terraform and have it provide outputs, have python generate Ansible hosts and vars from those outputs, and then run Ansible. What flavor of k8s are you looking at? k0s, k3s, microk8s, etc?

Ah, shortly after posting I found a clean solution. Adding --restart=Never makes the command work as expected. % time kubectl run -n kcox-test --image busybox --attach test --restart=Never -- date Thu Jan 27 15:10:41 UTC 2022 time Total: 9.449 (1%), User: 0.100, System: 0.034 - kubectl run -n kcox-test --image busybox --attach test --restart=Never -- date

If the application listens on loopback only by default it's ports couldn't be accessible by other pods. port-forward works because it ignores IP addresses and kubelet makes a proxy.

I've found the problem after some time, will add an answer here in case someone finds this in the future. Basically, my problem was exactly what is described in https://github.com/aws/amazon-vpc-cni-k8s/issues/1531 I had Calico installed in my cluster and was using a version that had a bug which blocked the Pod networking during the termination phase. This was the tutorial I followed when I installed Calico: https://docs.aws.amazon.com/eks/latest/userguide/calico.html All I had to do to fix the problem was to uninstall Calico and install it again using a newer version (by following this same tutorial again) Additional tip: One of the things that helped me debug this problem was to make sure my apps weren't the ones to blame on this behavior. I did this by running tcpdump in the Pod and forwarding its output to a local Wireshark for visualization. See this: https://downey.io/blog/kubernetes-ephemeral-debug-container-tcpdump/ After I found out my apps were sending traffic correctly, I knew the problem was something inside Kubernetes and started looking for possible culprits.

A helm hook pretty much has to be a Job or Pod. Thus choosing between using a hook or using a Job doesn't make much sense - you are usually best off using a Job which is annotated as a pre-upgrade hook. And yes, the timeout probably needs to be increased by the users. Unfortunately it's not possible to define the timeout in the chart itself. In some cases it makes sense to have a db upgrade happen after the helm chart has been applied and in that case it can be a "normal" Job. This pattern however requires coordination between the Pods accessing the DB and the upgrade process such as a table in the DB which declares the version(s) of the schema.

This is a little off. You are comparing a "Time Series Database" to a "Log Shipper". How Prometheus Works Any service/entity prometheus wants to run monitor has to provide an "exporter". An exporter is just a http endpoint that has metrics in a simple text format. The exporter can be a standalone app, a web sever linking to a file, or something like spring boot literally just adding a url to serve prometheus metrics. Lots of other systems can pickup and use this format, not just prometheus. You configure prometheus to "scrape" these metrics and store them in its own Time Series Database, usually at 30 second intervals. Then prometheus is basically your metrics database. It also integrates with alert manager to help you trigger alerts to slack, pager duty, etc. How Fluent Bit Works Fluent Bit is a log shipper. It is better compared with, say, logstash. It can basically take any kind of log data (text lines) and ship/route/filter/etc them to a destination system. It is not itself a time series database like prometheus though. Also prometheus is not a log shipper, it can't go make arbitrary routing decisions on the data being sent to it. This may be worth a skim - https://logz.io/blog/fluentd-logstash/ .

Actually, I should have searched more: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport is what I want, letting me expose ports on my local machine.

I suggest you create a Deployment of caddy:2 Pods which a Caddyfile mounted from a Secret, something like this: http://elastic-proxy.default.svc.cluster.local:9243 { reverse_proxy https://xxx.yyy.eu-west-3.aws.elastic-cloud.com:9243 { header_up Authorization "Basic Zm9vOmJhcgo=" header_up Host {upstream_hostport} header_up X-Forwarded-Host {host} } } The create a Service named elastic-proxy in the namespace (default in this example). Point the app Pods to this service instead of the Elastic Cloud. Explanation: This will create a Caddy 2 server which reverse proxies everything coming to port 9243 with virtual host elastic-proxy.default.svc.cluster.local to the Elastic Cloud and adds an Authorization header with the Basic Auth credentials (foo:bar here). You can create the password with base64 like this: $ echo 'elastic:password' | base64 ZWxhc3RpYzpwYXNzd29yZAo= Every time you change the password (why do you even change it?), just modify the Caddyfile in the Secret.

here is the solution I found in stack overflow https://stackoverflow.com/questions/52119985/kubeadm-init-shows-kubelet-isnt-running-or-healthy what I did here please create vi /etc/docker/daemon.json and inside added below: } "exec-opts": ["native.cgroupdriver=systemd"] } after run this command and solve the problem sudo systemctl daemon-reload sudo systemctl restart docker sudo systemctl restart kubelet

Add any tiny http-server wrapper to your script and deploy your docker image as a https://cloud.google.com/run and use https://cloud.google.com/scheduler that will invoke it by cron. https://cloud.google.com/run/docs/triggering/using-scheduler

I'll give some general suggestions and specific example how we are doing it - hope it will help. First of all, your idea to have a separate deployment repository is good and it is considered best practice in most cases. Next, it makes more sense to organize things on artifact level rather than code level - essentially, your deployment repository would have records of artifacts rather than code pointers via say git submodules. Personally, I try to stay away from git submodules whenever possible as they complicate things a lot. To elaborate on the above, each of your existing individual repositories would produce artifacts (docker images) via CI, place this artifacts to some registry, and then you would reference those artifacts in your deployment repository. Now, the basic ways of maintaining artifact changes in deployment repository is manual edits of artifact identifiers, which can be simplified by referencing via tags. I.e., you reference specific artifact as myartifact:2 - and then all version 2 artifacts would automatically make it to release. Say you start with 2.0.0, then add 2.0.1 - it will be picked accordingly. Note, that referencing via tags is fine for testing, but not recommended for production - for production we always recommend specific referencing via sha256 digest of the image which ensures uniqueness and immutability. To simplify the above tasks, people may put all artifact images into helm values files or use Kustomization with all image identifiers in one place. Some pull request logic may also be put on top (which is not always needed in the fully automated way below). Finally, there are fully automated ways. The way we do it is we use automation via Reliza Hub - tool we are working on - to glue everything together. So in the nutshell it tells your helm chart what artifacts to use in the automated way. Here is my toy project that shows various paths around the concepts above: https://github.com/taleodor/mafia-deployment

After talking with the support staff from the cloud provider, reading through the docs and looking at how other cloud providers do it, I believe that both of them are needed. In order to have a simple DNS zone management, you need to expose all your cluster nodes in some way and that way is with an external LoadBalancer. The LoadBalancer makes sure to always point to all the cluster nodes even when there are changes to the nodes themselves (add / remove). This way you can update the DNS zone for the domain you want by pointing to the LoadBalancer IP. Of course, you will need to make sure that this IP will not change. Since now there is a way to route external traffic from your custom domain to the kubernetes cluster, you need to know where to redirect that traffic. Here comes the Ingress Controller. With it you can forward traffic based on host name, for example, in order to reach your desired Service. Comparing two cloud providers and their approach for publicly exposing an application, I found that both of them are using an external LoadBalancer together with an IngressController: Scaleway In order to expose all cluster nodes through a LoadBalancer you need to create a Service with the following: spec: type: LoadBalancer ports: - port: 80 name: http targetPort: 80 - port: 443 name: https targetPort: 443 targetPort is required to be those exact ports. After LoadBalancer is created you can use its IP address for the DNS Records. Now, you can create the Ingress Controller to forward traffic to Services. AWS You can create an Ingress Controller which itself creates an AWS Application Load Balancer. The ALB does not have an IP, instead it relies on a CNAME Record. Both of them are using an external LoadBalancer to forward traffic to the cluster and both of them are using IngressController to redirect traffic to Services. I believe this is the way of publicly expose an application behind a kubernetes cluster.

Tumblr and Salesforce have open sourced their generic injectors: https://github.com/tumblr/k8s-sidecar-injector https://github.com/salesforce/generic-sidecar-injector Both of these should fit your simple use case but offer a lot more.

Related issue: https://github.com/awslabs/amazon-eks-ami/issues/636 So, not specifying --dns-cluster-ip, --b64-cluster-ca and --apiserver-endpoint fixed the DNS issue. Final bootstrap command: /etc/eks/bootstrap.sh exchange-develop --kubelet-extra-args '--node-labels=eks.amazonaws.com/sourceLaunchTemplateVersion=1,alpha.eksctl.io/cluster-name=exchange-develop,alpha.eksctl.io/nodegroup-name=custom-ami-bootstrap-ssh,eks.amazonaws.com/nodegroup-image=ami-0082fe83ca6d7cf7d,eks.amazonaws.com/capacityType=ON_DEMAND,eks.amazonaws.com/nodegroup=custom-bootstrap,worker=default --max-pods=110' --use-max-pods false

Jasper - I think you want to establish some form of communication between GitLab and your server, in particular, GitLab should be able to initiate events on the server in response to events (changes) in the GitLab repository. A common pattern for a solution is to use a webhook -- a feature of GitLab (I assume, I use GitHub) that allows you to instruct GitLab to perform a given action when something happens in GitLab. You can choose which things, for example, you might choose to have GitLab fire the webhook whenever there's a push. What happens when the push occurs? You need to pull from the repo, build a new container, upload it to the container repo, then run docker using the new container. The webhook is expected to be a URI that refers to a remote server ... in particular, the remove server needs to be listening on the port specified by the URI. The most common port is either the HTTP port 80, or the HTTPS port 443 -- and the reason they are popular is that many systems don't block them, as they do with most other ports. So you need something on your server listening on port 80/443 that will do these steps. So the steps needed (pull, build, upload, run) often fall into two phases: "build" and "run", where build includes all the steps needed to deploy a workable application. GitHub offers "github actions" -- I suspect GitLab has something similar -- that give you access to a temporary compute resource for things like running commands e.g. "pull", "compile", and "docker". Or, you could build a basic web server app that ran these commands. Either way, something on the server will need to run docker (or maybe ask it to restart) using the newly built image. Could you use Kubernetes for this ... sure ... but as you imply, it's too much for the single task. Learning how to deploy kubernetes is a lot of work. Instead, Google "xyx web server" where xyz is the programing language or environment to find a suitable installable server. Most offer a simple way to integrate system commands and respond to webhooks.

Solved updating the prometheus operator 0.38.1 -> 0.39.0. More details: https://github.com/apache/camel-k/issues/2794

Restarting the CoreDNS services has fixed the issue. Apparently there is some stale state it can get into. So if you see this behavior, a restart kicks it back into the right state. This is quite rare so it should be OK.

Flux is pretty smart and has some built it mechanism to avoid unnecessary updates. Helm has a version in the Chart.yaml, I'm assuming an inconsequential commit wouldn't update this version so flux might be assuming that it doesn't need to do anything. Changing the version would be something to try. To avoid the downtime problems, you need to double check your configs. Are you using the replace strategy to the install? Are you using force updates? Both of those strategies would delete the deployments before updating and that might be the cause of your downtime.

A bare pod and is exactly the same as a pod created by a deployment, the difference is that the deployment automates several tasks that you would have to execute manually. deployments, daemonsets, statefulsets, jobs and cronjobs are all pod managers, they all create/delete pods under the hood. Let's say you have a service sending requests to a pod running nginx containers that serve your site and you want to update the new container image that has the new files. steps: create a new pod with the updated image portforward to it and check if the pod is up and nginx is responding to requests add the labels that the service will look for delete the old pod sounds good? that's exactly what the deployment will do if you just update the image directly on it. You can have some resilience built into pods with pod lifecycle, which will attempt to restart the containers, but there's a limit to it, and managing changes and scaling pods manually is awful.