Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you may not be able to execute some actions.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

K8s cluster not deploying deployments across all the nodes

briley last edited by

I am a beginner in kubernetes. I have a cluster running with 4 nodes. All the nodes are currently running fine. They are not tainted, or drained. Every time I deploy a deployment it will only deploy it across 2 nodes instead of 4 nodes. I am trying to look for kube-scheduler logs, but because I am using Oracle Cloud OKE service (kubernetes service for Oracle cloud). They don't make kube-scheduler logs available to customers. What tests or what verifications do I need to do?
Reply Quote 0
1 Reply Last reply
N
Nykeriab last edited by

If you want to place a pod on each node you can use DaemonSet. DaemonSets are configured to distribute the application to each node. For detailed information;

https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/

You can find out if DaemonSet is what you need by looking specifically at this title;

https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/#deployments
Reply Quote 0
1 Reply Last reply

2
Posts

0
Views

Suggested Topics

Ingress vs Load Balancer
Continuous Integration and Delivery (CI, CD) • kubernetes ssl ingress load balance • • Mystic

2

0
Votes

2
Posts

0
Views

R

After talking with the support staff from the cloud provider, reading through the docs and looking at how other cloud providers do it, I believe that both of them are needed. In order to have a simple DNS zone management, you need to expose all your cluster nodes in some way and that way is with an external LoadBalancer. The LoadBalancer makes sure to always point to all the cluster nodes even when there are changes to the nodes themselves (add / remove). This way you can update the DNS zone for the domain you want by pointing to the LoadBalancer IP. Of course, you will need to make sure that this IP will not change. Since now there is a way to route external traffic from your custom domain to the kubernetes cluster, you need to know where to redirect that traffic. Here comes the Ingress Controller. With it you can forward traffic based on host name, for example, in order to reach your desired Service. Comparing two cloud providers and their approach for publicly exposing an application, I found that both of them are using an external LoadBalancer together with an IngressController: Scaleway In order to expose all cluster nodes through a LoadBalancer you need to create a Service with the following: spec: type: LoadBalancer ports: - port: 80 name: http targetPort: 80 - port: 443 name: https targetPort: 443 targetPort is required to be those exact ports. After LoadBalancer is created you can use its IP address for the DNS Records. Now, you can create the Ingress Controller to forward traffic to Services. AWS You can create an Ingress Controller which itself creates an AWS Application Load Balancer. The ALB does not have an IP, instead it relies on a CNAME Record. Both of them are using an external LoadBalancer to forward traffic to the cluster and both of them are using IngressController to redirect traffic to Services. I believe this is the way of publicly expose an application behind a kubernetes cluster.
F

GKE: Sharing the same filesystem across pods
Continuous Integration and Delivery (CI, CD) • storage google cloud storage • • fbio

2

0
Votes

2
Posts

1
Views

D

I think it's a pretty n00b question I suppose you'd need a pod which manages the filesystem, and make it available to the other pods which need the functionality to store data on it... Br, S.
E

Should I shut down the staging environment when the deployment test ends?
Continuous Integration and Delivery (CI, CD) • testing deployment amazon ec2 environments • • emran

2

0
Votes

2
Posts

1
Views

You can approach this case with multiple ways. For example you can set an auto-shutdown for 20:00 pm and auto-start for 8:00 am. Use Terraform to create your environment and, after deployment/testings, you can easily destroy it, saving your costs. And many other ways. I commented on paths that you can follow, so you should choose one that brings you greater benefit and agility both in creating your environments and in testing them. Regards
J

Interacting with a Trident container (NetApp) via CLI within a Openshift/Kubernetes cluster?
Continuous Integration and Delivery (CI, CD) • storage kubernetes openshift persistent volumes • • jules

2

0
Votes

2
Posts

1
Views

N

There's apparently 2 methods one can use to leverage tridentctl to interact with a running Trident Pod in their Openshift/Kubernetes cluster. 1. Server string The tridentctl CLI can be instructed to talk to whatever server you want remotely using the -s or --server argument. In this context you could use the approach of remote shelling into the Trident Pod and then directing it to interact with the local Trident Orchestrator daemon. For example $ oc -n trident rsh trident-6bdbdbc5dd-l8qtr Defaulting container name to trident-main. Use 'oc describe pod/trident-6bdbdbc5dd-l8qtr -n trident' to see all of the containers in this pod. / # Now to interact: / # tridentctl -s 127.0.0.1:8000 get backend +--------------------------+----------------+--------+---------+ | NAME | STORAGE DRIVER | ONLINE | VOLUMES | +--------------------------+----------------+--------+---------+ | ontapnas_192.168.101.101 | ontap-nas | true | 389 | +--------------------------+----------------+--------+---------+ / # 2. Creating a Bash Function To expand on the above method, one could construct a shell function or alias in Bash to streamline this interaction like so: Openshift tridentctl () { oc rsh -t -n trident -c trident-main \ $(oc get pods -n trident -l app=trident.netapp.io -o jsonpath="{.items[].metadata.name}") \ tridentctl -s 127.0.0.1:8000 $* } Kubernetes tridentctl () { kubectl exec -t -n trident \ $(kubectl get pods -n trident -l app=trident.netapp.io -o jsonpath="{.items[].metadata.name}") \ -c trident-main -- tridentctl -s 127.0.0.1:8000 $* } With one of these functions set, you could then run the following command from your macOS system: $ tridentctl get storageclass -o json { "items": [ { "Config": { "version": "1", "name": "basic", "attributes": { "backendType": "ontap-nas" }, "storagePools": null, "additionalStoragePools": null }, "storage": { "ontapnas_192.168.101.101": [ "NA_01_aggr1", "NA_01_aggr2" ] } } ] } How it works The above function merely determines the name of the Pod within the trident namespace and then remotely executes the command tridentctl using the same method above (from method #1) of connecting to the locally running Trident Orchestrator daemon within the Pod using the --server string. The advantage with one of these approaches is that you simply can use a function/alias to codify it. Caveats Keep in mind that with the above shell function/alias approach will not work with commands such as this: $ tridentctl create backend -f <backend-file> because the files you'll be referencing with it, -f ``` , will be local to the system where you'll be running the function/alias, but the tridentctl command will actually be executing within a Pod on another system. This Pod will not have access to your local system's filesystem. Your only options in this situation will be to either copy the file into the Pod's /tmp filesystem and execute it from there or to redirect the contents of the file, <backend-file> through a pipe, (|), for example: $ cat <backend-file> | tridentctl create backend -f -
K

When to put all containers in a pod and when to deploy them individually?
Continuous Integration and Delivery (CI, CD) • kubernetes • • Kadir

2

0
Votes

2
Posts

1
Views

P

Let's start from the beginning. First, I'll answer the original question, the one in the title. When to put all containers in a pod and when to deploy them individually? This depends on what your app does. For instance (this is an actual example from the company I work for), for a web application, you may have a pod which runs three containers: app: the actual app with its code nginx: serves all the requests heka: a log service which proxies the logs to wherever you want them The reason for grouping this three containers within a single pod is simple: a pod assures you the containers run in the same Kubernetes node, this is to have the least possible latency with requests being interchanged along the three containers. That being said, this is only an example, you could have different and multiple reasons for grouping containers in a pod, but I think this is the most common. With that being clarified, I follow to the question that you present in the body of your post. How do you access a single container within a pod? This is simply using kubectl, I use the command like this: kubectl exec -it -c= -- /bin/bash Suppose we want to access the nginx container inside our my-app pod; we would type this: kubectl exec -it my-app -c=nginx -- /bin/bash If you don't specify the container with the -c flag, it defaults to the first container that is in the Kubernetes yaml (I think). So when you say it was impossible to access a container, you actually did access a container (the default one), because a pod cannot be accessed, it is merely a collection of containers and its configurations. Hope I've been helpful.
Y

Does Buildbot support complicated build/deploy scenarios?
Continuous Integration and Delivery (CI, CD) • continuous integration continuous deployment toolchain • • yajahira

2

0
Votes

2
Posts

1
Views

C

Jenkins supports scripted pipeline which can handle my every requirement. Buildbot can handle these requirements with custom steps, i suppose, but it's too hard if compared to Jenkins' scripted pipeline.
G

evenly distribute pods over nodes
Continuous Integration and Delivery (CI, CD) • kubernetes azure kubernetes service • • guilherme

2

0
Votes

2
Posts

0
Views

T

Found the answer. When the deployment has a yaml with the following labels. kind: Deployment metadata: name: revproxy labels: app: proxy spec: selector: matchLabels: role: proxy template: metadata: labels: role: proxy The matchLabels in topologySpreadConstraints with matchLabels 'revproxy' is wrong. topologySpreadConstraints: - maxSkew: 1 topologyKey: kubernetes.io/hostname whenUnsatisfiable: ScheduleAnyway labelSelector: matchLabels: app: revproxy When pointing to just 'proxy' in the labelSelector the pods get distributed over the 3 nodes evenly. matchLabels: app: proxy Am still confused about the 3x the same labels in a description, changing any of these breaks the deployment, but at least it works.
A

Kubernetes - Deploying older version of app in new API
Continuous Integration and Delivery (CI, CD) • kubernetes • • Avante

2

0
Votes

2
Posts

1
Views

E

Prior to apps/v1 the selector field was implicitly picked up from the pod template's labels. So in your case it would be: spec: selector: matchLabels: io.kompose.service: web Starting with apps/v1 an explicit selector field is required by the Kubernetes API. I would suggest not to change the iptables on your own as it it the responsibility of your Kubernetes network plugin and is most likely done well. As far as I am concerned you may simple update the Deployment manifest and no uninstall is required.
Z

Kubernetes fails to do do garbage collection on images
Continuous Integration and Delivery (CI, CD) • docker kubernetes • • zymir

2

0
Votes

2
Posts

0
Views

G

I think I fixed the issue. It seems it comes up, when the kubelet service starts before the docker service is up and running. So I added the directive After=docker.service to the [Unit] block of the kubelet service and restarted it. The issue seems to be gone now.
T

How to get traefik ingress to load balance across several ClusterIP services
Continuous Integration and Delivery (CI, CD) • kubernetes traefik • • Trenton

2

0
Votes

2
Posts

1
Views

S

The sample I posted initially works as is: apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: my-service-ingress spec: entryPoints: - web - websecure routes: - kind: Rule match: Host(`myservice.mycompany.example`) services: - kind: Service name: myservice-node1-clusterip passHostHeader: true port: 80 - kind: Service name: myservice-node2-clusterip passHostHeader: true port: 80 - kind: Service name: myservice-node3-clusterip passHostHeader: true port: 80 - kind: Service name: myservice-node4-clusterip passHostHeader: true port: 80
Use one Helm chart for ALL microservices?
Continuous Integration and Delivery (CI, CD) • kubernetes helm kubernetes helm • • Laycee

2

0
Votes

2
Posts

1
Views

G

Generally, a Helm chart would cover all your microservices - ideally, you should be able to deploy whole application in one chart. It may get a little messy if you include external 3rd party dependencies, where you would need to handle dependent charts, but I assume this is out of scope of your question. More so, if your microservices are very similar you could write for loops, such as: {{- range $index, $service := .Values.myservices}} apiVersion: apps/v1 kind: Deployment metadata: name: myapp-{{ $service.name }} # --- etc --- {{- end }} Now, where to put your chart - in most cases a preferred way is to have a separate repo for all Ops CD files, including Helm chart - so this is where Helm chart would live. This is not to be mixed with CI files, such as Dockerfile - those should live alongside microservice repositories themselves.
How do you remove all amazon s3 files with jenkins before copying artifacts?
Software Programming • jenkins amazon web services deployment devops • • Demir

2

0
Votes

2
Posts

0
Views

E

aws s3 rb s3://bucket-name --forceSource: https://docs.aws.amazon.com/AmazonS3/latest/dev/delete-or-empty-bucket.html#delete-bucket-awscli
How to report Code Coverage metrics across git repositories?
Continuous Integration and Delivery (CI, CD) • azure devops teamcity code coverage • • Mystic

2

0
Votes

2
Posts

1
Views

If you use python and GitLab like I do, you can include a test coverage report in your CI/CD pipeline. You pip install coverage and then just run coverage in your pipeline. The following is an excerpt from an AWS lambda pipeline, but you should be able to find something similar for your environment: script: - pip install -r ./awslambda/requirements.txt - coverage run -m unittest discover -v -s "./awslambda" -p "*_test.py" - coverage report The above generates a nice code coverage report and it is free of charge, for Python, AWS and GitLab.
A

How to build, package and deploy a react-native-web app in a Docker container?
Continuous Integration and Delivery (CI, CD) • docker software engineering containers cicd • • Amritpal

2

0
Votes

2
Posts

1
Views

It's a pretty general question, I will try to answer the best I can: If you want to just build and run an image: Build the project: npx expo-cli init react_native_app --npm You need to create a Dockerfile file in your repository Docker build -t {tag_name} . (You can replace '.' eith the Dockerfile path) Docker run... If you want you can create docker compose file to better control your app. Reference: https://www.rockyourcode.com/how-to-run-react-native-expo-web-in-a-docker-container/
Does Kubernetes Ingress use NodePort?
Continuous Integration and Delivery (CI, CD) • kubernetes ingress • • Marcee

2

0
Votes

2
Posts

1
Views

K

For your first question: When using a Kubernetes LoadBalancer or Ingress, is the normal NodePort used as the underlying connection point? Technically it depends on the cloud network for LoadBalancers (e.g. Google Compute Engine does not technically require this although one may be created). However, I believe a NodePort and ClusterIP are generally created. For your question on the usage of NodePorts: The short answer is yes, they are safe to use if you know what you are doing. The longer answer is that NodePorts are great for debugging and development but have some issues in production: You can only have one service on a port. HTTP/HTTPS traffic may be routed through a non-standard port. The port will be opened on all nodes. However, NodePorts do provide some nice flexibility (from Kubernetes documentation Using a NodePort gives you the freedom to set up your own load balancing solution, to configure environments that are not fully supported by Kubernetes, or even to just expose one or more nodes’ IPs directly.

K8s cluster not deploying deployments across all the nodes

Suggested Topics

Ingress vs Load Balancer Continuous Integration and Delivery (CI, CD) • kubernetes ssl ingress load balance • • Mystic

GKE: Sharing the same filesystem across pods Continuous Integration and Delivery (CI, CD) • storage google cloud storage • • fbio

Should I shut down the staging environment when the deployment test ends? Continuous Integration and Delivery (CI, CD) • testing deployment amazon ec2 environments • • emran

Interacting with a Trident container (NetApp) via CLI within a Openshift/Kubernetes cluster? Continuous Integration and Delivery (CI, CD) • storage kubernetes openshift persistent volumes • • jules

When to put all containers in a pod and when to deploy them individually? Continuous Integration and Delivery (CI, CD) • kubernetes • • Kadir

Does Buildbot support complicated build/deploy scenarios? Continuous Integration and Delivery (CI, CD) • continuous integration continuous deployment toolchain • • yajahira

evenly distribute pods over nodes Continuous Integration and Delivery (CI, CD) • kubernetes azure kubernetes service • • guilherme

Kubernetes - Deploying older version of app in new API Continuous Integration and Delivery (CI, CD) • kubernetes • • Avante

Kubernetes fails to do do garbage collection on images Continuous Integration and Delivery (CI, CD) • docker kubernetes • • zymir

How to get traefik ingress to load balance across several ClusterIP services Continuous Integration and Delivery (CI, CD) • kubernetes traefik • • Trenton

Use one Helm chart for ALL microservices? Continuous Integration and Delivery (CI, CD) • kubernetes helm kubernetes helm • • Laycee

How do you remove all amazon s3 files with jenkins before copying artifacts? Software Programming • jenkins amazon web services deployment devops • • Demir

How to report Code Coverage metrics across git repositories? Continuous Integration and Delivery (CI, CD) • azure devops teamcity code coverage • • Mystic

How to build, package and deploy a react-native-web app in a Docker container? Continuous Integration and Delivery (CI, CD) • docker software engineering containers cicd • • Amritpal

Does Kubernetes Ingress use NodePort? Continuous Integration and Delivery (CI, CD) • kubernetes ingress • • Marcee

Ingress vs Load Balancer
Continuous Integration and Delivery (CI, CD) • kubernetes ssl ingress load balance • • Mystic

GKE: Sharing the same filesystem across pods
Continuous Integration and Delivery (CI, CD) • storage google cloud storage • • fbio

Should I shut down the staging environment when the deployment test ends?
Continuous Integration and Delivery (CI, CD) • testing deployment amazon ec2 environments • • emran

Interacting with a Trident container (NetApp) via CLI within a Openshift/Kubernetes cluster?
Continuous Integration and Delivery (CI, CD) • storage kubernetes openshift persistent volumes • • jules

When to put all containers in a pod and when to deploy them individually?
Continuous Integration and Delivery (CI, CD) • kubernetes • • Kadir

Does Buildbot support complicated build/deploy scenarios?
Continuous Integration and Delivery (CI, CD) • continuous integration continuous deployment toolchain • • yajahira

evenly distribute pods over nodes
Continuous Integration and Delivery (CI, CD) • kubernetes azure kubernetes service • • guilherme

Kubernetes - Deploying older version of app in new API
Continuous Integration and Delivery (CI, CD) • kubernetes • • Avante

Kubernetes fails to do do garbage collection on images
Continuous Integration and Delivery (CI, CD) • docker kubernetes • • zymir

How to get traefik ingress to load balance across several ClusterIP services
Continuous Integration and Delivery (CI, CD) • kubernetes traefik • • Trenton

Use one Helm chart for ALL microservices?
Continuous Integration and Delivery (CI, CD) • kubernetes helm kubernetes helm • • Laycee

How do you remove all amazon s3 files with jenkins before copying artifacts?
Software Programming • jenkins amazon web services deployment devops • • Demir

How to report Code Coverage metrics across git repositories?
Continuous Integration and Delivery (CI, CD) • azure devops teamcity code coverage • • Mystic

How to build, package and deploy a react-native-web app in a Docker container?
Continuous Integration and Delivery (CI, CD) • docker software engineering containers cicd • • Amritpal

Does Kubernetes Ingress use NodePort?
Continuous Integration and Delivery (CI, CD) • kubernetes ingress • • Marcee