Deploy the Kubernetes cluster#


Since Hub23 now shares the same Kubernetes infrastructure as the Turing Federation BinderHub, the following docs are only here for clarity and should not be acted upon without first notifying the operating team and removing the Turing’s BinderHub from the Federation.

Finding a Kubernetes version#

First, find a reasonably up-to-date version of Kubernetes to install onto the cluster. This can be done with the below command which will output a table as demonstrated.

$ az aks get-versions -l westeurope -o table
KubernetesVersion    Upgrades
-------------------  -------------------------
1.20.2(preview)      None available
1.19.7               1.20.2(preview)
1.19.6               1.19.7, 1.20.2(preview)
1.18.14              1.19.6, 1.19.7
1.18.10              1.18.14, 1.19.6, 1.19.7
1.17.16              1.18.10, 1.18.14
1.17.13              1.17.16, 1.18.10, 1.18.14

It is recommended to pick a version number that has at least one upgrade version available that is not in preview, so we can be sure that any bugs are likely to have been found and fixed. This would be 1.19.6 based on the above example.

Create the AKS cluster#

The following command will deploy a Kubernetes cluster into the Hub23 resource group. This command has been known to take between 7 and 30 minutes to execute depending on resource availability in the location we set when creating the resource group.

az aks create \
    --name hub23cluster \
    --resource-group hub23 \
    --kubernetes-version VERSION \
    --ssh-key-value .secret/ \
    --node-count 3 \
    --node-vm-size Standard_D2s_v3 \
    --service-principal $(cat .secret/appId.txt) \
    --client-secret $(cat .secret/appKey.txt) \
    --dns-service-ip \
    --docker-bridge-address \
    --network-plugin kubenet \
    --network-policy calico \
    --max-pods 110 \
    --service-cidr \
    --vnet-subnet-id ${SUBNET_ID} \
    --output table
  • --kubernetes-version: It’s recommended to use the most up-to-date version of Kubernetes.

  • --node-count is the number of nodes to be deployed. 3 is recommended for a stable, scalable cluster.

  • --node-vm-size denotes the type of virtual machine to be deployed. A list of Linux types can be found here.

  • --dns-service-ip: An IP address assigned to the Kubernetes DNS service.

  • --docker-bridge-address: A specific IP address and netmask for the Docker bridge, using standard CIDR notation.

  • --network-plugin: The Kubernetes network plugin to use.

  • --network-policy: The Kubernetes network policy to use.

  • --max-pods: The maximum number of pods per node. The Kubernetes API seems to struggle above 110 pods.

  • --service-cidr: A CIDR notation IP range from which to assign service cluster IPs.

  • --vnet-subnet-id: The ID of a subnet in an existing VNet into which to deploy the cluster.

Enable Autoscaling#

To enable autoscaling on the cluster, add the following flags to the above az aks create command.

  • --enable-cluster-autoscaler enables the Kubernetes autoscaler

  • --min-count/--max-count defines the minimum and maximum number of nodes to be spun up/down

Create multiple nodepools#

To create multiple nodepools within your Kubernetes cluster, add the following flags to the above az aks create command.

--nodepool-name { core | user }
--nodepool-labels{ core | user }
  • --nodepool-name assigns a name to your new nodepool

  • --nodepool-labels allows the definition of labels that will propagate to all nodes within the nodepool

It is recommended to use iether core or user to name/label these nodepools as BinderHub will try to schedule pods according to these labels.


The first nodepool create (i.e. with az aks create command) cannot be deleted as it contains the kubeadmin kubelet.

Another nodepool can then be added as follows:

az aks nodepool add \
    --cluster-name hub23cluster \
    --name user \
    --resource-group hub23 \
    --kubernetes-version VERSION \
    --max-pods 110 \
    --node-count 3 \
    --node-vm-size Standard_D2s_v3 \
    --vnet-subnet-id ${SUBNET_ID} \
    --labels \
    --output table


See the az aks nodepool docs for more info and commands.


The flags to enable autoscaling can also be used here.

Delete local copies of the secret files#

Once the Kubernetes cluster is deployed, you should delete the local copy of the Service Principal and public SSH key.

rm .secret/
rm .secret/appId.txt
rm .secret/appKey.txt

Get credentials for kubectl#

We need to configure the local installation of the Kubernetes CLI to work with the version deployed onto the cluster, and do so with the following command.

az aks get-credentials --name hub23cluster --resource-group hub23

This command would need to be repeated when trying to manage the cluster from another computer or if you have been working with a different cluster.

Check the cluster is fully functional#

Output the status of the nodes.

kubectl get nodes

All nodes should have STATUS as Ready.