Nick Sabine

How to Cluster Kubernetes on RHEL 7

In the last post, we installed Kubernetes locally on a single RHEL7 server. This time, we’re modifying the procedure to distribute jobs over two servers. The installation steps are exactly the same, up until we start Kubernetes, so I won’t rehash… see them here: http://nicksabine.com/post/kubernetes-rhel7

Instead of starting Kubernetes via script, we’re going to start via systemd.

In my configuration, I have the following VMs:

1
2

master=192.168.56.103
minion=192.168.56.102

Create all of the following files on the master. On the minion, create only the kubernetes-kubelet.service and kubernetes-proxy.service files.

/usr/lib/systemd/system/kubernetes-apiserver.service:

  1. [Unit]
  2. Description=Kubernetes API Server
  3.  
  4. [Service]
  5. ExecStart=/opt/kubernetes/output/go/bin/apiserver --logtostderr=true -etcd_servers=http://localhost:4001 -address=0.0.0.0 -port=8080 -machines=192.168.56.102,192.168.56.103
  6.  
  7. [Install]
  8. WantedBy=multi-user.target

Note: in the above, replace the comma-delimited list of -machines with the IPs of all the minions in the cluster.

/usr/lib/systemd/system/kubernetes-controller-manager.service:

  1. [Unit]
  2. Description=Kubernetes Controller Manager
  3.  
  4. [Service]
  5. ExecStart=/opt/kubernetes/output/go/bin/controller-manager --logtostderr=true -master=192.168.56.103:8080
  6.  
  7. [Install]
  8. WantedBy=multi-user.target

Note: in the above, replace the IP of the master

/usr/lib/systemd/system/etcd.service:

  1. [Unit]
  2. Description=Etcd Server
  3. After=network.target
  4.  
  5. [Service]
  6. Type=simple
  7. #etcd logs to the journal directly, suppress double logging
  8. StandardOutput=null
  9. WorkingDirectory=/opt/etcd
  10. ExecStart=/opt/etcd/bin/etcd
  11.  
  12. [Install]
  13. WantedBy=multi-user.target

/usr/lib/systemd/system/kubernetes-kubelet.service:

  1. [Unit]
  2. Description=Kubernetes Kubelet
  3.  
  4. [Service]
  5. ExecStart=/opt/kubernetes/output/go/bin/kubelet --logtostderr=true -etcd_servers=http://192.168.56.103:4001 -address=192.168.56.102 -hostname_override=192.168.56.102
  6.  
  7. [Install]
  8. WantedBy=multi-user.target

Note: in the above, replace the IP of the etcd_server (running on the master), and replace the -address and -hostname_override with the IP of the local minion server.

/usr/lib/systemd/system/kubernetes-proxy.service:

  1. [Unit]
  2. Description=Kubernetes Proxy
  3.  
  4. [Service]
  5. ExecStart=/opt/kubernetes/output/go/bin/proxy --logtostderr=true -etcd_servers=http://192.168.56.103:4001
  6.  
  7. [Install]
  8. WantedBy=multi-user.target

Note: in the above, replace the IP of the -etcd_servers (running on the master)

Start Kubernetes (on master)

  1. systemctl daemon-reload
  2.  
  3. systemctl enable etcd
  4. systemctl start etcd
  5.  
  6. systemctl enable kubernetes-apiserver.service
  7. systemctl start kubernetes-apiserver.service
  8.  
  9. systemctl enable kubernetes-controller-manager
  10. systemctl start kubernetes-controller-manager
  11.  
  12. systemctl enable kubernetes-kubelet
  13. systemctl start kubernetes-kubelet
  14.  
  15. systemctl enable kubernetes-proxy
  16. systemctl start kubernetes-proxy

Start Kubernetes (on minon)

  1. systemctl daemon-reload
  2.  
  3. systemctl enable kubernetes-kubelet
  4. systemctl start kubernetes-kubelet
  5.  
  6. systemctl enable kubernetes-proxy
  7. systemctl start kubernetes-proxy

References:

Show what’s running in Kubernetes (nothing yet!)

Tell the client where the server is running

  1. sed -i -e 's/^KUBERNETES_PROVIDER.*$/KUBERNETES_PROVIDER="local"/' /opt/kubernetes/cluster/kube-env.sh

Note: If you want to run kubecfg.sh on any host other than the master, you need to set the master’s location in /opt/kubernetes/cluster/local/config-default.sh:

  1. #IP LOCATIONS FOR INTERACTING WITH THE MASTER
  2. export KUBE_MASTER_IP="192.168.56.103"
  3. export KUBERNETES_MASTER="http://${KUBE_MASTER_IP}:8080"

List pods:

  1. /opt/kubernetes/cluster/kubecfg.sh list /pods

List services:

  1. /opt/kubernetes/cluster/kubecfg.sh list /services

List Replication Controllers:

  1. /opt/kubernetes/cluster/kubecfg.sh list /replicationControllers

Once you have this up and running, try deploying the GuestBook example: https://github.com/GoogleCloudPlatform/kubernetes/tree/master/examples/guestbook, but note that Kubernetes will not be able to deploy more than one of each type of pod per minion due to port conflicts. So if you are running a 2-minion cluster, the replicationController for the frontend needs to be modified to create 2 replicas, rather than 3.

comments powered by Disqus